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ABSTEACT 

The technical report has provided a functional 
description of an operational program planning and evaluation 
component called Operation Guidance, which is part of the Career 
Planning Support System (CPSS). (The CPSS, a systematic approach to 
improving high school career planning programs, has addressed itself 
to the critical career guidance deficiencies of career development of 
minority youth, career development of women, program planning and 
evaluation, and the transition from school to work.) Operation 
Guidance will provide a package of manuals, procedural guides, 
audiovisual aids, and inservice materials that are designed to serve 
as a tool with which individual high schools can design, implement, 
and evaluate their own career planning programs. Following an 
introductory section. Section 2 traces product development. Section 3 
describes methodology. Section 4 discusses design decision, and 
Section 5 describes the national field test. Field testing is 
scheduled from September 1974 to June 1976, with the entire CPSS 
materials scheduled for availability in late 1976. Approximately half 
of the document consists of supplementary appendixes, including 
Operation Guidance product elements, product and installation 
specifications, plan for behavioral objectives, product engineering 
phase position description, and national field test position 
description (school coordinator for Operation Guidance) • (EA) 
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The Center for Vocational Education's mission is to 
increase the ability of diverse agencies, institu- 
tions ^ and organizations to solve educational 
problems relating to individual career planning and 
preparation. The Center fulfills its mission by: 

Generating knowi§edge through research 

Developing educational programs and products 

Evaluating individual program needs and outcome 

Installing educational programs and products 
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PREFACE 



The challenge to assist youth in gaining the skills and knowledge to plan and execute meaningful 
careers is a primary concern of the educational community, particularly high schools. To help meet 
the challenge, The Center has been developing and testing a systems approach for high schools to use 
in delivering career development services to youth. The Career Planning Support System has two ma 
jor parts, operational planning and evaluation procedures, and innovative career guidance strategies 
for special populations. The development of the planning and evaluation procedures (referred to as 
"Operation Guidance") is the focus of this report. 

The primary intent of this technical version of the report is to provide other developers of educa 
tional products with a case study which describes in detail the building of a comprehensive system for 
solving a pervasive and chronic problem in American education. A general version of the. report, which 
is available from The Center, is intended to provide information about the Career Planning Support 
System to career education practitioners at all levels. 

We appreciate the contributions of the participants in the six schools that helped make this part 
of the Career Planning Support System a realistically developed and viable product. The many people 
to whom special appreciation is due are listed elsewhere in the report, but I would like to acknowledge 
here the contribution of the work unit staff headed by Warren N. Suzuki, and assisted by Richard P. 
Coatney, Harry N. Drier, and Paul E. Shaltry, and to Robert E. Campbell, program director. Career 
Planning Support System. 



Robert E. Taylor 
Director 
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I. INTRODUCTION 



The Problem 

The process of developing a satisfying career is a complex undertaking that young people, in 
particular, tend to stumble through at great expense to themselves and society. Many youth are in- 
adequately prepared for life roles they will assume in the community (National Advisory Council 
for Vocational Education, 1972, Ginzberg, 1971, as examples). Thus, despite its advances, the 
American educational system today is being harshly criticized for shortcomings in preparing adapt- 
ably useful and employable citizens. The problem has long range implications. Because of the 
changing nature of our society and the job market, young people not only need to make career de- 
cisions for their first years after leaving high school but also must be well enough prepared to plan 
for career changes throughout their lives. They need to know how to cope with change-to know 
the importance of continued learning and how to continue to learn and to be able to make and im- 
plement a series of career decisions compatible with their own potentials and desires. 

Traditionally, the major responsibility of helping students with career planning has been dele- 
gated to schools. Furthermore, the primary focus for delivering career planning assistance to students, 
traditionally, has been through guidance programs, more specifically, counselors. However, limited 
national assessments of career guidance programs, such as those conducted by Herr (1968), Ehrle 
(1969), Gelatt (1969), Rosen (1969), and Ginzberg (1971), have indicated that career guidance pro- 
grams have been neither effective nor efficient. 

Career guidance includes in its functional domain a wide assortment of services, ranging from 
personal counseling through information providing, to attendance keeping. Educational personnel, 
usually guidance counselors, have been developing career guidance programs based on prescriptions 
found in their professional literature, dicta emitted by district and state educational agencies, and 
pressures of other educational personnel in their schools. Fads and political expediencies have also 
dictated program content and configuration. 

Based on a national study, Campbell et al. (1968) concluded that the root problem with ca- 
reer guidance programs appears to be that high schools attempt to offer more career guidance ser- 
vices than they can effectively support with available and realistically obtainable resources. The 
study also indicated that only a few school programs have implemented recent innovations in career 
guidance methods and techniques. If student needs for career planning assistance are to be satisfied, 
then high schools should have career planning programs that have been developed both systematically 
and realistically. 

At this time, it is highly doubtful that a single, standardized program could be developed to 
meet the requirements of widely varied local conditions and student populations. However, each 
high school and/or school system should be able to develop a career planning program that effectively 
and efficiently responds to local conditions and unique student populations by applying systems 
technology. 
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National attention has been focused on the role of career guidance. Although career guidance 
is only a part of a troubled educational system, educators recognize its potential for becoming an 
effective means of helping all students in an important way. There appears to be a definite climate 
for positive change. 

Career Planning Support System 

The Center for Vocational Education (CVE) at The Ohio State University has undertaken the 
development of a systematic approach to improving high school career planning programs. This pro- 
gram, called the Career Planning Support System (CPSS), is sponsored by the National Institute of 
Education (NIE). 

The construction of the Career Planning Support System requires the progressive contribution 
of four parallel strands of research and development. Each strand addresses a critical career guidance 
deficiency. The four strands are: (1) the career development of minority youth, (2) the career de- 
velopment of women, (3) program planning and evaluation, and (4) the transition from school to 
work. 

Procedural Model 

This report deals with but one aspect of the CPSS, the product engineering of an operational 
program planning and evaluation component. The antecedents of this effort are based on a national 
survey and report of the status of career guidance (Campbell, 1968) and a feasibility study that in- 
cluded the development of a procedural model (Campbell, 1971). The niajor stages of the develop- 
mental effort are depicted in Figure 1. 

It is from the procedural model (feasibility study) that the product engineering phase took its 
direction in building an operational product for high schools. The procedural model is graphically 
presented in Figure 2 and was initially designed for use at all levels of public education. 

Parts of the procedural model were subjected to a simulated trial at a comprehensive, urban 
high school. The procedural model, per se, however, was not readily usable by school personnel at 
the high school level of public education since application of the model required specialized know- 
ledge of and skills in a variety of fields, such as career development theory, measurement, evaluation, 
and survey research, to name a few. 

Approach to Product Engineering 

The product engineering phase was conducted from February 1, 1971 to September 30, 1974. 
During this time the developers sought to build a relatively inexpensive package that could be used 
in comprehensive high schools across the country. Further, it was intended that the resulting pack- 
age of materials could be used without the help of the developers or others acting as consultants pri- 
marily due to the potential drain that external assistance can make on already hardpressed resources 
of high schools and sponsors of research and development. 
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Figure 2. Procedural Model for Developing Career Guidance Programs* 





CONTEXT EVALUATION 




PRIORITIES 




BEHAVIORAL OBJECTIVES 


1. Determine student needs 

2. Determine existing resources 

3. Translate needs into goals 


Assign priorities to 
program goals 


Derive behavioral objectives 
for goals assigned highest 
priorities 









METHOD SELECTION 



1. Identify alternative 
methods 

2. Establish selection 
criteria 

3. Select a method 





PROCESS EVALUATION 


T 


Determine if technique can function efficiently 


R 


within school and career guidance program 


I 

A 


PRODUCT EVALUATION 


L 


Determine if students are achieving 




the behav ^ral objectives 



RECYCLE 



Decide whether to develop service for 
goal assigned next highest priority or 
to start process over with a context 
evaluation 



*Campbell, Suzuki, Gabria, Jr., 1972. 





TECHNIQUE SELECTION 




PLAN TRIAL & 
IMPLEMENTATION 




1, Identify alternative 
techniques 

2, Establish selection 
criteria 

3, Select a technique 


1. Plan trial of technique 

2. Develop strategies to 
facilitate implementation 
of technique 







ADOPTION 



1. Phase out existing technique 

2. Phase ^n tested technique 



Overview of Remainder of Report 



The balance of this report deals with a functional description of the planning and evaluation cor 
ponent of the CPSS as it existed on the threshold of the two-year field test that began in September 
1974. Following the product description is an account of the methodology employed during the 
product engineering phase. This serves to acquaint the reader with the tools and techniques that 
were used to develop the product. The "Rationale for Design Decisions" section addresses the major 
changes made in the product as a result of trying elements in realistic settings. The last section pro- 
vides a forecast for the product as it relates to the development of the CPSS, the implications of the 
field test, and eventual availability. 
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ii. PRODUCT DESCRIPTION 



The product engineering phase of the Operation Guidance component of the CPSS generated a 
full complement of interrelated materials designed primarily for use in comprehensive high schools 
in the United States but, with modification, could be used in all the nation's high schools. What fol- 
lows is a brief description of the product, an overview of its process, and a description of how it 
works in the school setting. 

Framework 

The general framework upon which the program planning and evaluation component of the 
CPSS was developed is depicted in Figure 3. The ultimate purpose of the CPSS is to reduce the 
amount of career floundering now experie.-.ced by many people in the world of work (the ultimate 
target population). To ameliorate this problem, students (the intermediate target population) in 
high school need to attain career development competencies through, among alternative ways, a 
career planning program which is a compilation of resources (people, materials, equipment, and 
space) in the school and the surrounding community. The planning, organizing, directing, and con- 
trolling, or management of these resources could facilitate the effective and efficient delivery jf ser- 
vices to students. It should be noted, however, that the delivery model exists within an environment 
and that the environments across schools, although possibly having common elements, are in many 
ways unique. Although the commonalities should be addressed, the differences must, to the degree 
feasible, be accommodated. 

Resources for program operation coupled with program support for career planning are qualita- 
tively and quantitatively finite and determine the upper limits of the services that can be provided to 
students. The expectations of a career planning program are limited by the resources available, i.e., 
the quantity (number and time combined) of people, materials, equipment, and space of a certain 
quality for providing assistance to students. The resource base for a program can only be expanded 
through the reallocation of resources in the school at large or the greater community. Regardless, 
resources are equateable to dollars, and any reallocation of resources is ultimately a reallocation of 
funds within the school and the community. 

The availability of resources, although necessary, will not in itself result in a "better" program. 
Therefore, the optimal organization of existing resources (whether currently used or not) seems to 
be critical to the maximization of the capacity of a career planning program to enable students to 
attain the necessary, if not all of the desirable, career development competencies to function effec- 
tively in his/her future life. 

Population 

The Operation Guidance component of the CPSS has been designed for use by high schools. In 
order to acquire maximum benefit from the component, a school should have certain characteristics. 
In general, a using school should be stable and receptive and have the climate for change. A description 
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of some of the indicators of a desirable environment is given in Appendix D (pages 63-83). 
These characteristics, however, have not been fully verified. Also, it is doubtful that many schools, 
if any, possess all of the desired characteristics. Therefore, the degree to which the product is com- 
patible to a school is determined by the extent to which the school is able to adopt it. That is, a 
school will benefit from the product if it is ready and able to change. 

Specifications 

A variety of outcomes, both short and long range, may result from the successful application 
of Operation Guidance. Of course, the degree to which results become evident is dependent on the 
degree to which students, teachers, administrators, counselors, and community members become 
actively involved in and committed to effecting change through systematic planning and evaluation, 
in general, and the product, in particular. These results are summarized as follows: 

1. A display of valid data indicative of student career development needs; 

2. A comprehensive record of resources that are available in both the school 
and community to support career planning activities; 

3. A list of career development goals and the priority that each should have 
for implementation; 

4. A sufficient number of valid, behaviorally stated objectives indicative of goals; 

5. Instructional strategies and methods, called, "career development units," designed 
to enable student(s) to achieve a goal or a set of goals; 

6. A capability on the part of the school to plan, implement, review, evaluate, and, 
if necessary, revise career ^.anning activities (resources) over an indefinite period 
of time. 

The above results could occur within one school year. There are, however, long-range results 
that may not become apparent immediately. For instance, after two years of implementation, the 
product could enable a school to show whether any career development needs have changed. In 
time, by effectively using resources, the ultimate outcome-amelioration of student needs-should 
become apparent. 

The specifications for Operation Guidance, that is, the desired outcomes of and outputs from 
those in the instrumental population who use it, are listed in Appendix C. For developmental pur- 
poses, the specifications are classified into two types, product and installation. Product specifica- 
tions describe those outputs (e.g., documents) that the instrumental population in a participating 
school should produce if they want to make systematic changes. Installation specifications are con- 
cerned with those outcomes (changes in behavior) and outputs of the instrumental population that 
should enable their achievement of the product specifications. Although product specifications are 
related and integrated in the materials, the categorization was made to emphasize the importance of 
product adoption to the eventual success of a school's application of the program planning and eval- 
uation component. The importance of the bifurcation of the specification is discussed further on 
pages 23-26. 

g 
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An Innovation Package 



As an educational innovation, the product sets the climate for improving career planning pro- 
grams by providing a systematic yet practical set of procedures for improvement. The Operation 
Guidance process encourages administrators, teachers, parents, students, and community members 
to participate in defining local career development needs, restructuring priorities, and developing 
ways to help students attain career development skills. The extent of help any high school can give 
limited only by the resources available or potentially available to support student-oriented career p 
ning activities. 



Systems methodology is utilized in the product. The systems approach is a logical, efficient 
way to solve problems. Although used successfully in government and industry, the systems approach 
is relatively new to public education. The method should enable a school to organize career develop^ 
ment information and strategies into a manageable structure based upon current career development 
needs of students (Campbell, 1971). 

The product does not necessarily prescribe program content; however, it serves as a tool for de- 
signmg, implementing, and evaluating a school's own career planning program. The package, includes: 

1. Guides detailing procedures for collecting data from students, faculties, graduates, 
and parents and from the community and school and their resources, deriving pro- 
gram goals, formulating behavioral objectives, creating Career Development Units, 
and evaluating the career planning program. 

2. Survey questionnaires. 



3. Manuals that provide instruction for performing technical tasks. 

4. Supporting references (e.g., a handbook of career guidance methods). 

5. Forms for recording information, decisions, findings, etc. 

6. Synchronized audiovisual presentations for providing introductory 
information to the total package and its major tasks. 

7. A comprehensive preservice and in-service staff development program 
using simulation techniques. 

A more detailed list of product elements is given in Appendix C. 



The materials are written so a school may use them as fast as it deems possible and necessary. 
Tangible student activities, however, usually occur during the first school year of using the product, 
which was designed to be compatible with existing career development programs. Rather than ig- 
noring current efforts, the product provides a framevyork for using or modifying existing programs, 
if needed, and coordinating both new and existing efforts. 

Lasting and meaningful change rarely comes easily and quickly, and change resulting from the 
use of the product is no exception. Operation Guidance is a continuous, long-term process of 
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systematic improvement. The product is designed to enable a career planning program to keep pace 
with changes in the school, students, and the local and national community. Figure 4 is a graphic 
description of the general sequence of the major activities of the pUinniiui ond uvdiudtion coinponmil 
of the CPSS. 

Process of Operation Guidance 

The product has detailed procedures for organizing faculty, students, and members of the com- 
munity to accomplish programmatic development. The general tasks are: (1) identifying and docu- 
menting needs and resources, (2) stating, ordering, and selecting program goals, (3) deriving behav- 
ioral objectives, (4) developing and implementing career planning activities (career development units), 
and (5) evaluating the effectiveness and efficiency of the units and program. 

Identifying Needs and Resources 

Survey questionnaires are administered to students to determine their career development needs. 
More general questionnaires are also completed by parents, faculty, and recent graduates of the school, 
asking their opinions of the type of career development areas that the high school should emphasize. 
This information should enable the school to identify the strengths and weaknesses of its career plan- 
ning program. 

While surveys are being administered and tabulated, a task force collects information about the 
school and community that most likely will be of value to developing the career development pro 
gram. This information includes a list of resources (people, space, materials, and equipment) and 
funds to purchase resources, that could be used for career planning activities designed to ameliorate 
the needs of the students. 

Goals 

Identified student needs are translated into goal statements. These program goals state the 
skills, information, and attitudes students should have in order to choose and plan a career. Goals 
are derived from information collected in the sut veys, state and local educational requirements, ca- 
reer development activities already being used in feeder schools, and the school itself. 

Career planning activities are designed to help students achieve these goals. Sufficient resources 
will most likely not be immediately available to support all learning activities for all of the goals, 
therefore, the goals are ranked in order of importance based on available information, such as ap- 
parent magnitude of student deficiencies, mandates, parental preferences, etc. Some of the goals 
are then oelected for implementation. Usually resource^i art allocated to the most important goals 
first, then to other goals as resources become avai lable. Regardless, a goal assigned a higher priority 
should be fully supported before resources are allocated to one assigned a lower priority. 

A community based advisory committee reviews the goals, the priorities assigned to them, and 
the goals selected for implementation. Recommendations may also be made to improve the goals . 
Finally, the principal reviews and, if appropriate, confirms or reassigns priorities to the goals that 
should be implemented. 
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Behavioral Objectives 



Once goals are selected for implementation, planning begins for the learning activities designed 
to achieve the goals. A coordinator is chosen to oversee the development and implementation of ac- 
tivities for each goal that has been selected. With the assistance of at least two persons skilled in writ 
ing behavioral objectives, each coordinator writes the major behavioral objectives, called criterion ob- 
jectives, for each goal to be implemented. These objectives state what skills, knowledge, and attitudes 
students (who participate in the career planning activities designed for the goal) are expected to ex- 
hibit after participating. The team composed of the coordinator and two behavioral objective special 
jsts also derives enabling objectives which specify the skills, knowledge, and attitudes that students 
should have in order to attain criterion objectives. Also, it may be necessary to develop enabling ob- 
jectives, for each goal to be implemented. These objectives state what skills, knowledge, and attitudes 
the students (who participate in the career planning activities designed for the goal) are expected to ex- 
have implication for staff in service activities and a public information program, to name a few. 

The number, scope, and complexity of criterion and enabling objectives should give some indi- 
cation of the ways in which students could be introduced to the career planning activities. The ob- 
jectives may be appropriate for immediate introduction in a regular class. On the other hand, they 
may be comprehensive enough to indicate that a separate course is needed to achieve a goal or a set 
of goals. Exploring alternatives is Encouraged, both within and outside the existing curriculum. Pro- 
posed objectives undergo a review process prior to their final approval to assure their validity and de- 
sirability. 

Methods 

The next stage of Operation Guidance transforms plans into action. Although behavioral objec- 
tives define what students are to achieve, schools have to choose ways for the objectives to be achieved. 
Most likely there will be more than one way to achieve each objective. The task is to select instruc 
tional and/or counseling methods that are appropriate to the individual or groups of students partici- 
pating in the career planning activities. In addition, the methods chosen should use a school's resources 
efficiently. 

Career Development Units 

All goal related information is collected into a formal plan for a career development unit{s). 
A career development unit (CDU) is a plan.ieu learning activity that attempts to enable students to 
achieve one or a logical set of career development goals that have been written for the program. The 
plan developed for a career development unit identifies the behavioral objectives that have been de- 
rived for each goal, the instructional methods and resources that will be used for attaining each ob- 
jective, and the manner in which the CDU will be installed into the school. This career development 
unit plan is subject to the principal's final approval with interim review by the Steering and Advisory 
Committees. 

Evaluation 

Three levels of evaluation occur. 
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1. At the completion of and, if desirable, during each career development unit, the partici- 
pating students critique the usefulness of the activity for attaining each behavioral objective and 
whether the activity was interesting. The person in charge of the career development unit assesses 
the extent to which each behavioral objective of the unit was achieved by the students involved. In 
addition, a leader summarizes the students' evaluation and describes the advantages and disadvantages 
of the activities designed to achieve each objective. 

The information obtained from this evaluation can be very helpful in indicating revisions that 
might be made to increase the effectiveness of the career development unit. Adjustment should be 
made to reflect these changes. 

2. An annual review is conducted of all career development units that have taken place during 
the year. The result of this review might mean a greater number of students would be involved during 
the next execution of the career development unit, changes could be made in the way resources are 
allocated, etc. 

The Operation Guidance process continues to improve a school's program by increments, devel- 
oping career development units as long as unmet needs exist and resources are available. Evaluation 
facilitates the determination of the continued validity of the program as it is used and grows. 

3. The third level of evaluation is called "recycling." The recycling process was designed to 
increase the probability that the career planning program continues to be effective in meeting stu- 
dents' needs, since the career development needs of students may change over time. Therefore, every 
two years the surveys are again administered to first- and last-year students of the school, their parents, 
the school/staff, and recent graduates. 

Information reflecting the needs of first-year students and those in the twelfth grade are recorded 
on graphs each time the surveys are administered (every two years). These graphs should indicate 
whether students' need for a given skill increases, decreases, or remains stable. Several adjustments 
in the career development program could result from this information. The priorities assigned to goals 
could be changed; the behavioral objectives for a given goal could be revised; and/or a career develop- 
ment unit could be expanded, decreased, or eliminated. 

Roles for School Personnel 

The success of a developmental project in a given school is dependent on the .cooperation of in- 
dividuals. To facilitate this cooperation, the materials suggest positions^and roles to be assumed by 
people in the school and in the community. The positions suggested are flexible as to how a school 
wants to fill them. The foundations of the planning and procedural component of the CPSS encour- 
ages differentiated staffing in career planning activities in the school. Certainly, counselors play a 
key role, but the overall effort can be enriched by judiciously combining the skills and talents of 
many people. The following is a brief description of the project positions. 

The two leadership roles called for by the program rest with the principal and the project coor- 
dinator. The principal is the key person in establishing the commitment that a school and community 
makes toward the successful conduct of the project. As the chief executive officer, the principal is 
ultimately responsible for all that happens in the school, including the success or failure of the project. 
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In addition to the leadership shown through overt commitment to the project, however, the degree 
to which a principal directly manages the project or delegates responsibility and authority is his/Kar 
own decision. 

In addition to the principal, the project coordinator needs to have a firm grasp of the scope and 
intent of Operation Guidance. It is recommended that the project coordinator be given at least half- 
time for the day to-day management of the project. The coordinator serves as the link between the 
principal and project activities. It is recommended that he/she also serves as the Steering Committee 
chairperson. As a change agent, the coordinator should be able to work well with all groups m order 
to be effective. The position description for a coordinator is given in Appendix G. 

The School's Cost 

Personnel . Many people should be encouraged to participate in the project. One advantage of 
large scale participation is to keep each individual's time expenditure at a reasonable level. In addi- 
tion, involving a cross section of individuals could result in greater faculty and student commitment 
to the project and the resulting upgraded career planning program. 

The approximate number of positions and the estimated time commitment that incumbents in 
any of the many positions would need to spend during each year of a project are as follows; 

1. Five to seven Steering Committee members (three to five faculty/staff members 
and two students) at an average of one and one-half hours per week. 

2. Forty task force members (sixteen to twenty faculty/staff members and eighteen 
to twenty student members) at an average of two hours per week for eight to ten 
weeks. 

3. A number of CDU coordinators equivalent to the number of active CDUs. 

An individual may occupy more than one position. Also, total number of people could be scaled 
downwards as the number of students in the school decreases below 600. 

Supplies , The supplies and equipment listed below will be needed during the project's first year. 

Additional expendable supplies will be needed during subsequent years in which the package is used, 

but at a considerably lower level. 
•« 

1. Printing (offset or comparable method) - the number of pages will be approximately 
8.5 times the number of students in the school. 

2. Duplication - 200 copies, preferably by electrostatic process. 

3. Postage (if surveys are mailed) - approximately $100. 

4. Office supplies (excluding paper for reproduction) - two reams of letterhead and 
1,400 business-size envelopes imprinted with the school's name. 
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5. Equipment - a 35 mm circular slide projector with remote controls that, 
optionally, is coupled with a slide-sound synchronizer capability. 

Staff Development 

Staff development is an integral part of the product. The various subtasks are explained and con- 
structed in a semi-programmed instructional format. This way a task may be executed and, at the 
same time, the performer can be learning, for example, survey research techniques. Another mode of 
staff development is used for special skills such as writing technically sound program goals and behav- 
ioral objectives. Staff development materials ^re also provided so a simulation of the project can oc- 
cur before actually implementing the project in t'le school. / 

Organization 

Several groups are formed on both a temporary and permanent basis. The following is a descrip- 
tion of committees, roles, and functions. 

Steering Committee . The Steering Committee is composed of teachers, counselors, and students, 
and has three, functions: (1) manages the project, (2) serves as a major decision making body, and (3) 
acts as representatives of the school population. 

As a managerial body, the Steering Committee is responsible for coordinating the many activities 
that are part of the project. Its duties include making certain that resources needed to carry out activ- 
ities are available. It monitors the work of task forces to be certain that tasks are completed appro- 
priately and in as short a time as possible. It also serves the function of coordinating tasks performed 
by these different work groups. The committee has the responsibility of storing all data and products 
of the project in such a manner that information may be located easily as it is needed. 

The Steering Committee is a major decision-making body. It has the responsibility of assuring 
that the upgraded program reflects the career planning needs of all students in the school. The Steer- 
ing Committee writes goals for the program and decides the order in which goals will be carried out, 
based on information collected from the school and community. The Steering Committee reviews 
behavioral objectives and plans for career development activities that are made to achieve these goals 
to provide assurance that they are complete, accurate, and appropriate. The committee submit re- 
commended goals, behavioral objectives, and plans for career development activities to the principal. 

There are usually five to seven members in the Steering Committee in addition to its chairperson, 
depending on the number of counselors in the school. It was assumed that the number of counselors 
in a school would be indicative of the potential resources available for career planning activities and 
the priority currently assigned to the career development of students. Also, the expertise of guidance 
counselors was needed for a variety of tasks and, therefore, it was necessary to allocate this possibly 
limited resource. As representatives of the school. Steering Committee members are encouraged to 
seek out others' advice as they work on the tasks described previously. 

Task Forces. Task forces are work groups usually composed of students, teachers, and counsel- 
ors. These groups are formed on a temporary basis to perform certain duties. Different task forces 
are formed to collect and tabulate survey data, to collect information about the school and community. 
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and to assist with the writing of behavioral objectives for program goals. The number of people on 
a task force depends on the activity to be completed. 

Data Collection and Tabulation Task Force . It is suggested that the Steering Committee chair- 
person act as the needs assessment leader for this task force. Duties include initiating and monitoring 
the activities of the various task force teams during the collection of survey data, receiving and storing 
data collected from surveys, assigning data tabulation tasks to team members and monitoring their 
work during data tabulation, and properly storing the surveys and end products for safe keeping when 
all work has been completed. 

i" 

Six or seven faculty/staff members (depending on the number of grade levels in the sch /ol) should 
serve on the task force. Their duties include helping to prepare and collect surveys and overseeing the 
process of transferring information from surveys to appropriate data summary tables. Fourteen to 
sixteen students, depending on the number of grade levels in the school, should also serve on the task 
force. Their duties include helping to prepare and collect surveys and transferring survey information 
to appropriate data summary tables under the supervision of faculty/staff members. 

School and Community Description Task Force . Duties of the chairperson for this task force 
include initiating and monitoring the tasks involved in the collection of background information and 
available resources in the school and community and keeping an up-to date record of the resources 
available for career development activities and how they are used. It is strongly suggested that the 
chairperson be drawn from the Steering Committee to facilitate the coordination of the project's ac 
tivities. In addition to the chairperson, two faculty/staff members and four student members should 
serve on the task force. Their duties include collecting information about the characteristics of the 
school and community and listing resources available to upgrade the school's career planning program. 
This information is recorded and stored in a loose leaf binder entitled, "Career Development Data." 
Unlike the chairperson of the task force whose responsibilities exist throughout the project, members 
of the task force only serve during the initial compilation of information and as needed during recycl 
ing of the school's career development program. 

Bahavioral Objectives Task Force . The chairperson of this task force should also be a member 
of the Steering Committee. It is suggested that the chairperson be skilled in the writing of behavioral 
objectives. His/her duties include initiating and monitoring the training and activities of the members 
of this task force and assisting with the writing of behavioral objectives for learning activities that are 
to be developed and implemented. 

Five faculty/staff members should also become behavioral objective specialists. It is suggested 
that these individuals, if possible, be knowledgeable in the writing of behavioral objectives prior to 
selection for the task force. Their duties include becoming familiar with the format of behavioral 
objectives recommended by the materials and then assisting with the writing of behavioral objectives 
for career development learning activities that are to be developed and implemented. 

Method Analysis Specialists . Two individuals are selected to serve as method analysis speciialists. 
It is suggested that one of the two specialists be a member of the Steering Committee. These individ 
uals become familiar with current career guidance methods by studying available references. Their 
main responsibility then involves meeting with faculty members serving as career development unit 
coordinators and aclvisors to assist them with selecting appropriate and effective methods for use in 
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achieving each of the behavioral objectives written for their career development units. Although they 
may be inactive most of the time, these specialists will be called upon to assist with method selection 
for each career development unit that is developed over time. They may also be asked to assist in re- 
vising methods used in career development units. 

Career Development Unit Coordinators and Advisors. Learning activities (career development 
units) are designed to help students acquire the skills, knowledge, or attitudes necessary to achieve 
each goal written for the career development program. The people responsible for developing these 
activities are called career development unit coordinators and advisors. 

The principal and Steering Committee should select an individual with expertise in the subject 
matter related to the goal or goals to be implemented to serve as a CDU coordinator. Primary respon- 
sibility for planning these learning activities for the goal rests with this individual. The duties of this 
individual include: 

- Becoming familiar with the school and community characteristics identified 
in the Career Development Data binder. 

- Writing the primary behavioral objectives, called criterion behavioral objectives, 
for the goal with the assistance of behavioral objective specialists. 

- Together with CDU advisors (if any), deriving the remaining behavioral objectives 
(enabling objectives) needed to achieve the goal with the assistance of behavioral 
objective specialists. 

- Working with the resource assessment leader, method analysis specialists, and 
CDU advisors to identify appropriate methods and resources needed to implement 
a career development unit. 

- Conducting evaluations of a career development unit. 

- Possibly serving as a career development unit advisor and implementing a career 
development unit. 

Career Development Unit Advisors . If the career development unit should be implemented in 
several areas or classes so that as many students as necessary will have an opportunity to participate 
in the career planning activities or in an area in which the coordinator does not have responsibility, 
advisors should be nominated to assist with the implementation of the career development unit. 
These individuals would work directly with the students, providing learning activities that will enable 
them to achieve the behavioral objectives and goals defined in the career development unit. Their 
specific duties involve: 

- Working with the career development unit coordinator and Behavioral Objective 
Task Force members to derive enabling behavioral objectives for the assigned goal. 

- Together with the resource assessment leader and method analysis specialists, 
identifying appropriate methods and resources needed to implement the career 
development unit. 
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- Organizing resources (equipment, personnel, materials, and space) needed for 
the activities involved in the career development unit. 

- Implementing the career development unit. 

- Evaluating student achievement of the behavioral objectives of the career 
development unit. 

Advisory Committee 

The Advisory Committee is composed of five to nine representatives of the community. This 
group acts as a liaison between the school and the community and provides recommendations con- 
cerning the planning and evaluation of the career development program. The Advisory Committee 
also serves as a source of information about the students' community environment. 

The principal should act as the chairperson of this committee. His duties wowld include presid- 
ing over meetings of the Advisory Committee. 

It is strongly recommended that the Steering Committee chairperson serve as executive secre- 
tary of the Advisory Committee. His/her duties would include notifying members of meeting dates, 
keeping records of the proceedings of Advisory Committee meetings, and corresponding with the 
members of the groups as needed. 

The membership of the Advisory Committee should consist of five to nine representatives of the 
business and educational community, governmental and civic groups, and parents. Their responsibil- 
ity includes providing information to the Steering Committee and reviewing outputs of the program, 
such as goals and behavioral objectives, and making recommendations for their improvements. 

A school may already have several advisory committees working in different areas. It may facil- 
itate the coordination of and communication among activities if members of some of those commit- 
tees are on the Advisory Committee for the project. Therefore, it may be beneficial for a school to 
consider individuals serving on other advisory committees when identifying and selecting members 
for the Advisory Committee. 

If at all possible, the Advisory Committee should have at least one each of the following types 
of individuals as members: 

A. Executive officer of a business-industrial concern or organization. 

B. Executive officer of a labor organization or union. 

C. Executive officer of the school's parent or parent- teacher association. 

D. Director of guidance and/or career education (or his/her representative) 
for the school district or county. 

E. Representative of one of the major community action groups. 
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Other types of individuals that may be appropriate for membership on the Advisory Committee 
are as follows: 

1. Faculty member of a university or college, preferably a specialist in career 
guidance and/or education. 

2. Representative of the state department of education's agency concerned with 
career guidance and/or career education. 

3. Director of vocati^.ial education for the school district or the state department 
of education. 

4. Representative of state department of labor or manpower or the state employ- 
ment service. 

5. Representative of a local office of the U.S. Department of Labor or Health, 
Education, and Welfare. 

6. Representative of a junior high school or middle school from which students 
transfer to the high school. 

This has been an overview of Operation Guidance, the operational planning and procedural parj 
of the CPSS, and, implicitly, its process. It is the version derived from many revisions based on ex- 
perience in the six high schools widely distributed across the country that participated in the product 
engineering of the product. The procedural model remained fairly well intact although some steps or 
nrodules were combined for efficiency in production and application. The procedures to operate the 
product in a school were built to accommodate realistic conditions and constraints. Without the help 
of the six product engmeering schools, the product could not have been developed. Once the product 
was complete, the developers embarked on a two-year national field test of the product which is de- 
scribed in the last section of this report. 
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ill. METHODOLOGY 



A. General Process 

The product engineering of CPSS (Operation Guidance) was a relatively large scale and complex 
effort. Initial planning called for the use of multiple developmental stages and phases in addition to 
the integration and utilization of outputs from other activities within and outside of The Center for 
Vocational-Education. 

Figure 5 depicts the general process which was used to develop the operational planning and 
evaluation component. It also shows two other units that provided major inputs to the component. 

During product engineering, the procedural model was implemented through the development 
of detailed procedures and supporting materials which are usable by comprehensive high schools. 
During initial prototype development, seven modules* containing procedural guides, instructional 
manuals instrumentation, etc., following guidelines in the procedural model were developed. The 
product elements or materials that comprised each module were then subjected to a series of trial 
and revision cycles. These phases were: (1) conducting a field trial of a module at a comprehensive 
high school; (2) revising the module based on evaluative data collected during the trial; (3) conduct- 
ing a second field trial of the module at another high school; and (4) again revising the module. The 
series of progressive, overlapping phases comprised the formative product engineering stage (Findlay, 
1973) of the unit. 

The product was then subjected to summative product engineering. Essentially, this was a trial 
of all^product elements as an entire system at four comprehensive high schools. Evaluative data con- 
cerning the functioning of the entire system were collected. Based on this information, the system 
was again revised, resulting in the product described in Chapter II of this report. 

Conceptually, formative product engineering was concerned with the internal validity and usable- 
ness of individual elements and modules of the planning and evaluation component. Summative pro 
duct engineering, on the other hand, primarily focused on the internal validity of the entire planning 
and evaluation component, including the completeness of the materials and procedures for installation 
and operation of the component. Therefore, the initially developed prototype was progressively re- 
fined through the successive field trial and revision phases. It should be noted that the magnitude of 
the revisions to the component did not progressively decrease during each successive phase, as was 
predicted during the initial planning of the work unit, but, in fact, increased as more evaluative data 
became available to the developmental i.eam. However, it should also be noted that the basic design 
concept remained relatively stable throughout the product engineering stage. 



♦ 

In the final revised product, the term "module" was not used. Rather, the tasks were organized by function 
and assigned the name of that function, e.g.. Resource Assessment, Data Collection and Tabulation. In all, there were 
eight functions, including orientation/organization. 
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Figure 5. General Product Engineering Phases and Concurrent Activities 
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Phases of the product engineering effort were overlapped to reduce the total developmental time 
for the operational planning and evaluation component of the CPSS. This strategy involved taking 
risks that could have been avoided if a linear sequence of product engineering phases was used. The 
most likely risk, and one that eventually became a reality, was concerned with the rate at which the 
schools using the product elements during the different field trial phases would progress. Participat- 
ing schools could and did use the materials at their own pace. The rate of progress for a school 
seemed to be related to the conditions within the school and community that facilitated or hindered 
the adoption of the product, coupled with the degree to which the materials (and, implicitly, the 
strategy) were refined. School(s) in later field trials tended to progress at a more rapid rate than the 
school(s) in earlier phase(s). This phenomenon was primarily attributed to the refinements in the 
product elements used in the succeeding field trial phases, including those specifically designed to 
reduce the amount of time needed to use the planning and evaluation component. Therefore, the 
lead time between field trial phases progressively decreased during the work unit. This eventually 
eliminated the time needed for collecting evaluative data and making the indicated revisions to the 
product elements. Although the progressively decreasing lead time between field trials was antici- 
pated, the rate of change was not expected. The second of two formative product engineering field 
trials was eliminated during the evaluation of the materials concerned with enabling the instrumental 
population to derive behavioral objectives from program goals and the site was redesignated a sum- 
mative product engineering field trial site. 



Although information from many sources, such as Herr and Cramer (1972), Hosford and 
Ryan (1970), Cook (1971), Stufflebeam (1968), and Super (1967), was used during the develop- 
ment of the operational planning and evaluation component, two are of special note. First, a hand- 
book of career guidance methods (Campbell, Walz, Miller, and Kriger, 1973) was designed for in- 
clusion in the component. Second, a case study of the adoption of Operation Guidance that was 
conducted concurrently with the product engineering work unit by another program within The 
Center for Vocational Education, provided evaluative data for revising the planning and evaluation 
component (Kesterand Howard, 1975), 

The remainder of this chapter is concerned with the major activities accomplished during the 
product engineering phase. Since some of the major activities were similar although differing m 
focus, the discussion will be divided into initial prototype productions, field trials of the prototype 
and collection of evaluative data, analysis of results and revising the component, and resources used 
during the product engineering phase. 

B, Initial Prototype Production 

The general procedure used to produce the initial prototype component is graphically shown in 
Figure 6. However, it should be noted that some of the product elements were not subjected to all 
of the developmental tasks due to time constraints during the initial prototype production and/or 
the two or three revisions. Trade offs were made between perfection and fulfilling the delivery ex- 
pectations of the schools participating in the work unit as field trial sites. That is, a balance had to 
be struck between personnel in a participating school losing interest in the use of the product because 
of inadequacies, such as typographical or technical errors, and not receiving materials so that they 
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Figure 6. General Procedure for Production of the Product 
(Only Major Path Shown) and Its Relationship to Field Trials 
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could move on to the next task. In addition, decisions to by-pass developmental steps were compli- 
cated by the developmental team's concern for the internal validity and the eventual generalizability 
of the product. 

The primary source for the development of the product and installation specifications (out 
comes and outputs) for the planning and evaluation component was the procedural model developed 
by Campbell (1971). Campbell's model laid the general foundations and described the general pro- 
cedures for systematic planning and evaluation of career guidance programs. Initially, the specifica 
tions were stated in relatively general terms, then progressively were made more detailed during each 
revision cycle as uncertainties were alleviated. 

Product and installation specifications are interdependent, and it was often difficult to decide 
whether a desired outcome or output should be classified as a product or installation specification. 
However, the bifurcation of desired outcomes and outputs, that is, the identification of the indicators 
of desired terminal states and the indicators that enable attainment of the terminal states, facilitated 
the evaluation of the product's performance. 

The distinction between product and installation outputs and outcomes also served to highlight 
the importance of the adoption of the planning and evaluation component by the instrumental pop- 
ulation. That is, concerns, such as the cost of purchasing and using the product by a school and the 
ease, or difficulty, with which the instrumental population is able to accomplish the many planning 
and evaluation activities, were as instrumental to the success of the product engineering effort as were 
the technical considerations of program planning and evaluation. From the perspective of evaluating 
and revising the product, too, observations on the achievement, or non-achievement, of installation 
specifications should provide insight or rationale for less than desired achievement of product speci- 
fications. 

Figure 7 graphically illustrates the different levels of the problem and associated target groups 
that could have been addressed by Operation Guidance (and, ultimately the CPSS) and, therefore, 
the levels at which the product should be assessed. The paradigm also shows the potential impact 
which the product (or, for that matter, any externally developed product) could have on the ultimate 
problem, occupational floundering. Implicit to the hierarchical arrangement of the problem is the 
reduction of deficiencies at lower levels before any long term amelioration of the problem at any next 
higher level is possible. 

Ideally, the target population should have consisted of, intermediately, students in high schools 
and, ultimately, graduates. The high schools may have three (10-12) or four (9-12) grade levels and 
are located throughout the United States. The impact, the reduction of floundering behavior, on this 
target could not, however, be readily or fully assessed for two main reasons. First, there was a lack 
of adequate theory and technology to do so, e.g., extant vocational or career development theories 
were not sufficiently specific and validated to serve as a basis for program development or assessment. 
Recently emerging technologies for the assessment of student achievement of career development 
behaviors are clear examples of these deficiencies. Second, the state of the art for evaluative meth* 
odology and, ultimately, research methodology was inadequate for assessing whether the reduction 
in the ultimate floundering behavior in students is attributable to a single treatment, such as a career 
guidance program. Further, those methodologies were inadequate for accomplishing such an assess 
ment within the time period for and resources of the product. It is doubtful that the effects a career 
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guidance program could have on an individual as opposed to the combined effects of the remamder 
of a school's program, the student's home life, and other external considerations such as the labor 
market, could be adequately isolated. Even if it were possible to isolate the effects of all other in- 
dependent variables, it could require, at a minimum, a longitudinal study to determine the degree to 
which a career guidance program would impact on an individual. 

Since the two major constraints above could not be overcome, it was necessary to focus at an 
intermediate level or levels, i.e., assessing program characteristics and the behavioral change of an 
instrumental population or subpopulation with the hope that these characteristics can contribute 
to the reduction of the ultimate problem. Even the former level of assessment involved risk; that is, 
possible failure to show effects, since the degree to which the environment could be controlled by 
an externally developed product is, realistically, relatively low. 



C. Collection of Evaluative Data 

In order to determine the effectiveness and efficiency of the operational planning and evaluation 
component, or, more aptly, modifications that needed to be made to the evolving packa^,-*, six compre- 
hensive high schools located in five states, their school districts (local education agencies), and state 
educational agencies voluntarily participated in the work unit as field trial sites. To the degree feasi- 
ble, the participants were located throughout the continental United States, among other consider 
ations, to facilitate the dissemination of the product during subsequent stages of the CPSS program. 
However, two schools were located in one local educational agency at the request of their local and 
stats educational agencies. The six schools, in alphabetical order, are: Agua Fria Union High School, 
Avondale, Arizona; East Bank Senior High School, East Bank, West Virginia; Walter F. George High 
School, Atlanta, Georgia, Jesse H. Jones Senior High School, Houston, Texas; Sunset High School, 
Beaverton, Oregon; and Booker T. Washington High School, Atlanta, Georgia. The length of partici- 
pation for these six schools was as follows: 

1. First formative product engineering field trial site — three academic years. 

2. Second formative product engineering field trial site — two and one-half 
academic years. 

3. Four summative product engineering field trial sites - two academic years. 

The six schools may generally be described as follows: Three of the schools had grades nine- 
twelve and three had grades ten-twelve. The average number of students for the six schools was 
1,41 1, ranging from 890 to 1,950. The overall racial/ethnic composition of the students was 53 
percent white, 42 percent black, and 5 percent from other groups. The average number of teachers 
in a school was sixty -six, ranging from forty to ninety three. The average number of counselors was 
six, ranging from two to twelve (average student/faculty ratio was 21:1, the average student/coun 
selor ratio was 235:1). All six schools were in Standard Metropolitan Statistical Areas (SMSAs). 
Three of the schools were inner urban, one in the suburbs, and two in the outlying area of the SMSA. 

The following is a general description of their participation in and the procedures for the collec 
tion of evaluative data on the different versions Of the planning and evaluation component. Before 



27 



proceeding, however, it should be noted that practical considerations of working with high schools 
and local and state educational agencies who expended relatively large amounts of their resources to 
use the evolving product often outweighed the need or desire for methodological elegance during the 
successive rounds of evaluation and revision of the planning and evaluation component. Although 
the use of field trial sites was to collect evaluative data from the development team's perspective, the 
schools perceived their participation in the work unit as a means to upgrade their career guidance 
programs. Therefore, the developmental team was obligated to provide first-hand assistance to a 
participating school to accomplish prescribed planning and evaluation tasks when the elements of 
the component were not able to do so. The evaluative data received from the participating schools 
also were at times less than ideal because of the additional resources that would have been needed 
to collect the desired information. 

Site Identification and Preparation 

Extensive lead time was needed to identify, select, and prepare the comprehensive high schools 
who served as the product engineering field trial sites. Although the amount of time varied signifi- 
cantly, approximately nine months were needed from initial planning to have a site ready for initial 
participation in the field trials. Potential field trial sites were identified by inquiries received from 
schools and local and state educational agencies regarding participation in the work unit. These in- 
quiries were generated by varying means, including a summary report of the procedural model 
(Campbell, 1971b). Other means of dissemination, such as a journal article (Campbell, Suzuki, and 
Gabria, 1972) and papers presented at meetings of professional organization, resulted in more in- 
quiries from interested agencies in the field. The inquiring organizations were considered as potential 
trial sites. Based on readily available information, each candidate was compared to a set of criteria 
developed by the development team. In general, these criteria were concerned with the following 
general conditions: 

1. The school should be located in a Standard Metropolitan Statistical Area (SMSA). 
Available demographic information would be more extensive for an SMSA than 
non-SMSA areas. 

2. The school should have at least 200 students per grade level, two or more 
counselors, and forty-five or more teachers. Although schools with smaller 
populations could probably use the evolving product, the development team 
felt that it would be more advantageous to minimize the amount of work that 
any individual would have to do during the field trials. 

3. The students, teachers, counselors, and administrators in the school and their 
local and state educational agencies should be committed to the participation 
in the field trials. The participants needed to commit a relatively large amount 
of resources in terms of time, space, equipment, and materials to their use of 

the evolving package. Also, malfunctions in the product were inevitable regardless 
of the effort of the development team to minimize them. 

4. The school should be a relatively stable setting in which to evaluate the performance 
of the evolving product. Instability, such as teacher or student unrest, would make 
the collection of evaluative data more difficult than it already was. 
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5. The school should be readily accessible by commercial air or private ground 
transportation. This condition was of particular concern for formative product 
engineering field trial sites due to the relative crudeness of the initial versions of 
the evolving planning and evaluation component. The development team there- 
fore needed the capability to access a site with minimal lead time needed should 
a serious problem with the use of the product arise. 

6. The participants should be willing to discuss with visitors their participation in 
the work unit. 

In addition to identifying candidates that were deemed best suited to serve as field trial sites, 
alternate sites were also identified in the event that a candidate was found to be unsuitable ur un 
willing to participate in the project. 

The school district and state department of education in which each candidate site resided were 
contacted to ascertain their continued willingness to participate in the field trials under the conditions 
specified by The Center and to acquire permission to conduct an initial site survey. An initial visit 
was then made to each site. During this initial visit, key decision-makers in the state and local educa- 
tional agencies were provided information on the field trial and key personnel were identified and 
contacted to provide input for joint agreements. An agreement was then developed for each field 
site. 

Site Survey 

Baseline data on each field site were collected for recycling and implementing decisions and/or 
consumer information. A portion of the information was acquired through the needs and resource 
assessments that were conducted by personnel at the field sites during their normal use of the com 
ponent. Instruments and procedures for these assessments were developed and validated prior to 
and during formative product engineering. 

Additional baseline data were collected after all agreements had been signed. During this site 
survey, information was collected on the following, characteristics of the student population (e.g., 
racial and ethnic composition, social environment, academic grouping), characteristics of the school 
(number of grades, curricula, demographic characteristics of the surrounding community, managerial 
structure), and faculty and staff characteristics (number and type of faculty and staff members' ex 
periential background, such as experience with behavioral objectives). 

Site Preparation 

Personnel at the field sites were briefed by the development team on their part in the field trials. 
A general briefing was given to the entire faculty, student leaders, and staff at each test site. Special 
briefings were given by the team to members of the working groups (Steering Committee, Advisory 
Committee, and task forces) at each site. These working groups were briefed immediately after their 
formation and prior to their initial use of product and installation materials and procedures. 

Procedures 

During the product trial, the product was used by personnel at the field trial sites with as little 
intervention and assistance by project personnel as possible. The development team attempted to be 
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as unobtrusive as possible during the field trial in order to minimize contamination of the evaluative 
data; however, team members observed the performance of the personnel in the schools while they 
used the product and installation packages. The team also interviewed selected personnel and stu- 
dents at the field trial sites and received all outputs developed by the participating schools as a nor- 
mal function of using the evolving product. 

Field Associates 

A field representative, called a field associate, was located in each field trial site. The field asso- 
ciates were employees of the local educational agencies in which they worked, but were, in general, 
released full time to assist The Center for Vocational Education with the field trials, i.e., they had 
clearly identified time to participate in and take directions from the development team for all matters 
related to the field trials. A position description for the field associates is given in Appendix F. Pro- 
cedures for the identification, selection, and hiring of field associates depended on the agreements ne- 
gotiated with and the personnel policies and procedures of the participating school districts, but the 
following general procedures were used: 

1. The participating school districts identified potential applicants for the field 
associate positiofis using criteria provided by The Center. 

2. The team leader or his representative interviewed the applicants and identified 
the candidates best suited for the positions. 

3. The team leader recommended to the participating school districts the candidates 
be placed in the field associate positions. 

The field associates received training prior to taking an active role in the work unit. After initial 
preparation, the field associates required additional preparation in order to maintain operational ef- 
fectiveness. As an example, during the initial preparation periods, the field associates became famih 
iar with the procedural model, its general purpose, the planning and evaluation component, and the 
evaluation plan. During the conduct of the field trials, field associates became conversant with the 
actual content and configuration of the total product and each product element. Initially, care was 
taken to insure that the field associates performed as members of the development team and not as 
a part of the school's staff, since a field associate could become co-opted by the school and, could, 
even unintentionally, invalidate evaluative data. Toward this end, local educational agencies were 
encouraged to place individuals who were not a member of the participating school's faculty in the 
field associate positions. However, this view changed when it appeared that individuals familiar to 
the faculty in a school took less time to become operationally effective in the school than those who 
were unfamiliar to the faculty. At times, too, the field associates were encouraged to become co- 
opted by the participating school to facilitate the use of the evolving product by the school. 

Instrumentation 

Evaluative data were collected in a variety of ways. The primary source of information on the 
effectiveness of the operational planning and evaluation component of the CPSS was the actual out- 
puts produced by members of the instrumental population in the participating high schools as a result 
of their normal use of the evolving package. These outputs are identified in the product specifications. 
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such as program goals, behavioral objectives, career development unit plans, etc. Formats or forms 
were provided in the packages that the participating field sites used to facilitate their documentations 
of results and conclusions. Also, other information, e.g., students' needs for career development, 
community and school resource characteristics, etc., which they documented in order to acquire the 
primary outputs was also provided to the developmental team for evaluative purposes. 

Criterion referenced tests were also built into printed instructional materials. In general, printed 
instructional materials contained self-administered, diagnostic pretests and mastery posttests. Direc- 
tions for administering and scoring the posttests were also contained in the package. 

As mentioned previously, another program The Center for Vocational Education conducted was 
an in-depth case study of the adoption of the operational planning and evaluation component of the 
CPSS as an innovation (Kester and Howard, 1975). Much of the information that they needed for 
their study had already been or was in the process of being collected by the project team for the pur 
pose of revising the product. Additional information that the staff of the case study collected was 
provided with recommendations to the development team for the planning and evaluation component. 
This latter information was concerned with issues such as the perceived complexity of the innovation 
and the acceptance of the program planning and evaluation component. 

Other means of collecting evaluative data are listed below: 

1. Product Memorandum were used by field associates to document apparent 
deficiencies in or possible improvements that can be made to product elements. 

2. Project Logs were used by field associates to/ecord project activities at their 
sites. The Project Log consisted of the Daily Log and Weekly Activity Summary. 

3. Time Logs were used to collect information on the amount of time an individual 
in a participating school spent working with the program planning and evaluation 
component. 

4. Budget Report was used to document monetary expenditures, especially local 
contributions, made by each field site. 

5. Individuals using the materials were encouraged to comment directly on product 
elements regarding their opinions on the materials. 

6. Observations by the development team members, including field associates, of the . 
actual use of the product elements provided information on the performance 
effectiveness and efficiency of the product. Although initially using interview 
schedules, the interviewer investigated in-depth system deficiencies that became 
apparent. When subjects at the sites completed major tasks, they were interviewed 
by team members. For each product element, special instructions on the collection 
of evaluative data were developed for the field associate. When feasible, different 
team members or more than one team member visited each participating school to 
facilitate the collection of reliable information through observational techniques. 

7. Meetings of working groups were tape-recorded (audio) when feasible. 
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Ideally, it would have been most desirable to assess every product and installation specification 
with the most sophisticated techniques available. However, it would require extensive developmental 
and administrative time and effort to do so. It would have also required an unreasonable amount of 
time for the subjects to participate in the resulting data gathering effort. Therefore, assessment was 
subjected to "cost-benefit" considerations, just as all other developmental activities were. 

The use of only two field sites during formative product engineering severely delimited the gen- 
eralizability of the findings. The product was and still is highly complex and contained a large num- 
ber of product elements. System malfunctions were expected even though much effort was expended 
during initial production and revision to make the product elements as effective and efficient as pos- 
sible. 



D. Revision 

The revision of the evolving operational planning and evaluation component of the CPSS in- 
volved the analysis of evaluative data and then the use of the findings to make appropriate modifi- 
cations to the package using the general procedures graphically shown in Figure 5. (p. 22 ). The eval- 
uative data and information received from the field trial sites were compiled by the development 
team members in a variety of ways. As an example, numerical data were tabulated, and observational 
data summarized. However, the raw and summarized information was stored until ready for use. 
Questions that were asked during the analysis of data were as follows: 

1. Has evaluative information been adequately collected, documented, and validated? 
Has evaluative information been utilized? Has the data been collected and stored 
in a timely, efficient manner? Has the field associate been co-opted? 

2. Have the product and/or installation specifications been met? If yes, was it met 
in the most efficient and effective manner feasible? 

3. Does the product element or set of product elements function as designed? 

4. Is the system, subsystem, or product element malfunction localized (i.e., a problem 
unique to one site) or is it generalized (i.e., a problem encountered at all sites)? 

5. How disruptive did the system, subsystem, or product element malfunction? 
Can the people at the site tolerate the malfunction? 

6. Are the product and/or installation specifications valid? 

7. Is the procedural model or the implementation of the procedural model valid? 

8. What are the consequences (e.g., continued cooperation from school, district, 
and state) if the modification is or is not made? 

9. What is the estimated cost for making the modification? 
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a. What manpower resources would be needed? 



b. What support services would be required? 

10. Will the modified product element require fieid trial? If so, how should it be retried 
and how much will the retrial cost? 

11. What are the consequences if the field trial is terminated before all product elements 
have been tried? 

In addition to the development team's analysis of the evaluative information, several critical 
decision points which occurred during the work unit were selected as candidates for external evalu- 
ation through The Center for Vocational Education's Assessment of Internal Decision Event (AIDE) 
strategy. The Center's Evaluation Division developed a separate detailed plan for an external review 
of the evaluative information. An example of a plan is given in Appendix D. Three outputs from the 
schools participated in the summative product engineering field trials were selected for external re- 
view. These were; (1) statement of student needs for career development, (2) behavioral objectives 
derived from program.goals, and (3) career development unit plans. The former two primary and 
their associated outputs were subjected to external review. The last outputs from the school were 
not reviewed by external evaluators and only by the development team because of the lack of time 
to do so. That is, revisions on the product elements concerned with the development, implementation, 
and evaluation of career development units were being finalized for the product field test of the com- 
ponent before an external review could be organized. 



IV. DESIGN DECISIONS 



At the beginning of the product engineering phase the developers had as a starting point the 
procedural model that is discussed on page 2 of thib report. The objective was to develop a useful 
product that generated student-oriented results in the high school. 

In the realm of "procedural politics" all dilemmas between methodological elegance and prac- 
ticality were decided in favor of practicality or what was perceived to be relevant and practical by 
the instrumental population. Concerns such as the amount of time available to accomplish tasks by 
and the varied levels of expertise within the instrumental population, adminictrative constraints, 
leadership style, etc., indicated the need for a highly detailed package. Throughout the product en- 
gineering phase there appeared to be the positive relatioriship between the amount of detailed ma- 
terial and the efficiency and effectiveness of the proyram developed by the instrumental population. 

Another general concern that occupied the product engineering phase was the ethical commit- 
ment on the part of the developers towards the field sites. In that the six schools had committed 
many people and considerable resources to the field trials, the developers felt obliged to point out 
potential pitfalls and, if necessarv, suggest remedies to problems that were directly associated with 
the product. It should be noted that in all cases advice from the developers was not obligatory. 
Final resolutions to problems rested with the school. The developers felt it *as more realistic, when 
necessary, to help the schools get results with the product rather than insist o, "pure" data from the 
field thus risking abortion of the field trial at a given school. 

The remainder of this section will highlight the iriajor product design concerns and considerations 
during the evolution of the field test product, hi general, the data received from the four to five sum* 
mative product engineering field sites indicated that the rriajority of the materials and procedures were 
effective and efficient. These findings were confirmed by external evaluators. Although such positive 
information was gratifying to the developers, of greater significance was the negative information that 
indicated a need for product modification. To the degree possible the product malfunctions were 
ameliorated. However, not all indicvited changes were made due to limitations on project resources 
and the time needed to affect the changts. For example, a resource pool of career development be- 
havioral indicators and a full computer program for processing survey data were not developed. 

The major product design concerns are iri the following categories, specifications, organization, 
needs assessment, resources, behavioral objectives, CZ\J plans, installation, and field personnel. 

Specifications 

The product and installation specifications evolved over the duration of the project. Regeneration 
resulted in greater comprehensiveness, specificity, and sophistication of the specifications. Admittedly, 
early specifications, especially product speciflcatiorib, were overly general and ambiguous. There are two 
primary reasons for this. First, less than nine months were available for the initial planning and the devel- 
oping of first materials needed for field trials. This was coupled with a low level of resource allocation to 
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the project during this period. Second, there was a suspicion among the developers that they did not 
know enough about high schools, the way they operated, their inhabitants, and the pressures and 
constraints placed upon them. Most critical was the potential for change which was directly related 
to each school's uniqueness. Therefore, there was a concern about spending too much time on plan- 
ning the product with a distinct possibility that modification of the initial conceptualiiCuiion was in- 
evitable, coupled with a concern that overly specific specifications would become "chisled in granite" 
and would be difficult to change in response to the field experiences. 

The current product specifications denote significant changes from the earlier versions in that 
there is greater specificity for both the product and installation concerns. Basic concepts and general 
outcomes did not change over time even though they became more concrete. As part of the develop- 
ment of product specifications, often installation concerns outweighed actual product concerns in an 
attempt to keep the field trials working. As a check on the validity of the evolving specification, the 
developers, periodically, had both the set and parts reviewed by external evaluators. 

Organization 

Initially the planning, evaluation, and installation tasks were organized linearly and grouped into 
modules. (The number of modules varied from five to ten.) However, evaluative data tended to in- 
dicate that completion of modules rather than the key outputs described in the product specifications 
became the goal of the working units within the school. Also, the modular design appeared to work 
against the continuity of the process. Schools seemed to develop a "modular mentality" rather than 
realizing the modules were but a part of a larger effort. Therefore, in the final version all references 
to modules were eliminated, and the process focused on tasks. On the other hand, the modular de- 
signation and linear development was a useful managerial device for the developers given the amount 
of time and resources available to the development staff at CVE. 

A major complaint from nearly all Steering Committee members of the participating schools 
was the amount of time that transpired between initiation of the effort and the first "concrete" re- 
sults. In the final version of the product the concurrent accomplishment of tasks by the instrumental 
population was used to a greater degree than initial versions. Although somewhat increasing the dif- 
ficulty of coordinating the work being done in the school, it was hoped that the accomplishment of 
concurrent tasks would accelerate the process and, therefore, be more conducive to maintaining and 
building interest in the use of the product. However, the degree to which concurrent tasks were de- 
veloped was limited by the amount pf project resources and available developmental time. If time 
had permitted, a special option would have been developed that would enable the school to imme- 
diately upgrade one or two elements of their existing career development program, so that positive 
results could be shown quicker during the first year. At the same time, other task forces or working 
groups would progress through the normal evaluation decision making process that is prescribed in 
the final version of the product. 

Another way in which this problem was addressed was to delegate many task force responsibilities 
to the Steering Committee to promote continuity and coordination of the effort. For example, the 
early procedures called for an ad hoc group to interpret survey data into goals. In the current version, 
the Steering Committee, the group with greatest product knowledge, addresses that task. 



36 



Needs Assessment 



One of the most difficult tasks during product engineering was developing materials and proce- 
dures for the instrumental population to collect information on student career planning and develop- 
ment needs. A search of the literature indicated that in assessing the career development behaviors 
of students, such as those being conducted by Crites (1969), Super (1957), and Westbrook (1971), 
were under way and had promise for inclusion in the product. However, it was Jater learned that the 
instrumentation that was being developed was inappropriate for program development because of a 
variety of reasons, such as inappropriate population and insufficient specificity of the information 
provided, including inappropriateness of the scoring (norm-referenced as opposed to criterion refer- 
enced). 

Because of the need for a data base upon which to build the program and, when necessary, to 
"prove" that student needs were not being satisfied (i.e., a "baseball bat" to get the attention of 
teachers, parents, and administrators), project staff with the assistance of the participating schools 
developed a set of assessment instrumentation. Following are some experiences and observations 
from the development of the instrumentation. 

Career development skills for students may be technical or non-technical. Technical skills are 
most often associated with occupational education. Non-technical career development skills can be 
displayed as in Figure 8. 

Experience at the field sites led the developers to believe that the information collected on the 
surveys mu:t be specific enough to yield concrete goals. Self-awareness, socioeconomic, and decision- 
making and planning skills that were included in the earlier versions of the survey instruments seemed 
to be difficult for the instrumental population to develop into student oriented, non process kinds 
of goals. Other skills appeared to be more specific in terms of what the students could or could not 
do, thus providing a more concrete base for goal development. Additionally, since the subject mate* 
rials were directed towards the high school it seemed more reasonable that career implementation 
skills were of a higher priority, particularly for juniors and seniors. Thus, the five areas of career de- 
velopment skills addressed in the surveys were: (1) occupational information, (2) educational infor- 
mation, (3) educational placement, (4) school and community resource exploration, and (5) job 
placement. The areas of self information, socioeconomic information, and decision-making and 
planning, though not included in the surveys, were considered enabling skills that students could ac- 
quire regardless of the goal selected. The area of non-technical vocational adjustment had only one 
goal representing it and so was included in the area of occupational information on the survey. 

There are basically five ways of obtaining information from students regarding their career de- 
velopment needs. The ideal method is to assess through actual performance whether the student can 
perform a certain task such as decision-making. Second, the student can be asked what he or she 
has done. Third, the student can be asked whether he/she can perform a task or can acquire certain 
information. Fourth, the student can be asked whether he/she needs assistance with a specific task 
or certain informatiori. Fifth, the student can be asked whether he wants assistance with specific 
tasks or certain information. The third level of assessment, that is, student's self report on whether 
he/she can do specific tasks or has certain information was used as a means of collecting information 
upon which to plan a school's career guidance program. Time and resources available to the project 
staff prevented the development of instrumentation at the first level, the assessment of actual student 
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performance. Initial versions of the instrumentation focused on a student's self report of felt needs 
and wants or desires for assistance. However, information gathered during interviews with students . 
who had responded to the instruments tended to indicate that a significant number of students did 
not indicate a need for a skill or information or desire for assistance if they did not know about 
the necessity for the skill or information. Although this was at times a function of wording and/or 
ordering of the items, interview data tended to indicate that some students did not want to state a 
need or desire for assistance if it was to be provided by a school. This was particularly evident when 
the assistance was related to social or personal problems. All in all, in the earlier versions of the in- 
strumentation items often appeared statistically reliable yet on further evaluation through individual 
interviews the information collected was often apparently unreliable. 

Admittedly, it may not be possible to formulate a truly comprehensive career development pro- 
gram with a survey of just a limited number of skills. It should, however, function as the basis or 
foundations for a comprehensive program. The skill areas were also incorporated to help organize the 
program while the specific skills in each area would be used to determine student deficiencies. Another 
reason for focusing on skills is that knowledge changes more rapidly than skills. Any knowledge test 
would have to be continually updated. 

The relative brevity of the questionnaires for collecting information from students, faculty /staff, 
parents, and graduates is a function of the amount of time that would be needed by them to complete 
the instrument. As an example, when administered to students it was highly desirable to have the 
complete administration time, that is, from the time they are released from classes to the reading of 
instructions, the responding to the instrument, collection, and sending the students back to the room, 
fall within a school period of approximately fifty minutes. Teachers did not appear to appreciate 
having students participate outside of this time construct since it tended to cause confusion in the 
school. As far as teachers, parents, and graduates, the longer the instrument the less likely they 
would respond to it. Also, the developers found that if the student survey was too long, then unre- 
liability appeared to be greatly increased. 

Terminology was also a problem. Many students equated jobs and occupations. As for careers, 
only "doctors" and "teachers," in other words those who go to college, had careers according to stu- 
dents. As a result, only the word "job," the most generally understood term, was used on the student 
survey. In the tabular display of data, the correct terminology was reinstated. 

Data Collection 

In the early administration of the survey, a systematic sample of students was drawn. However, 
experience demonstrated that tapping the whole student body facilitated administration. Since all 
students are involved, surveys could be administered in homerooms or a comparable common period. 
Sampling students would cause commotion in singling out students, not to mention the problems of 
following them up if they missed the initial administration. Another consideration for surveying the 
entire student body was to encourage student interest in career development activities. 

The developers wanted to create a computer data processing subsystem for the needs assessment 
but did not have adequate resources to do so. Written procedures for manually tabulating data were 
given first priority since many schools do not have computer capabilities. The procedures are arranged 
to yield survey item responses in an eighty column format. If a school has computer facilities available, 
then it should be quite easy to adapt the survey data for computer processing. 
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Goals 



Probably the most important issue in the needs assessment part of the product was the translation 
of data into goals. Initially, the developers used a dual approach for this translation. The instrumental 
population was given a highly structured decision-logic table to arrive at goals for half the survey items. 
For the other half, they were presented the data and with little decision-making structure were asked 
to draw conclusions from the information. While the highly structured approach generally yielded 
more valid results, the committees were very uncomfortable handling data in a highly structured way 
for a variety of reasons, including information not accounted for by the product. Given no structure 
to handle the data, illogical need statements often concluded from the existing information. To facil- 
itate the probability that logically derived goals would come from the data, the developers sought to 
strike a compromise between a highly structured decision-logic table and no structure. This resulted 
in the survey items being stated so a "yes" or "no" response was required. The items were so worded 
that a "no" response indicated a self-report of a career development deficiency. Thus, goal statements 
could be derived by simply rewording the survey item. 

Resources 

The issue of resources available to support program development was and is of prime concern. 
Since a basic tenet was the efficient use of resources, initially, schools were encouraged to collect 
and consider all resources when making decisions about priorities and selecting goals for implemen- 
tation. This process tended to create an information overload for those trying to assign priorities to 
pals. This overload appeared to result in a reversion to biases or personal concerns rather than selec- 
tion of priorities based on available data. As a compromise, procedures were revised to consider rank- 
ing goals based only on the survey data. A relatively simple accounting structure for resources was 
devised to facilitate allocation of resources and is brought into the implementation process rather 
than when priorities are selected. This was done to distribute the use of information more evenly 
throughout the developmental process. 

Behavioral Objectives 

The behavioral objectives section of the product presented content and procedural problems. 
Instructional objectives relating directly to career development behavior of students were not avail- 
able. A framework for constructing technically accurate, student oriented behavioral objectives was 
provided for the instrument?^l population. Examples for the instrumental population to follow were 
career development oriented. Had there been sufficient resources, a pool of career development ob- 
jectives would have been developed to facilitate the writing of objectives in the school. 

Procedurally, the first way objectives were written for the school was by an ad hoc task force 
that wrote all objectives for all goals. This tended to create a quality control problem in that the 
task could be very overwhelming for one group. Also, those charged with the implementation of the 
objectives tended to feel that those objectives were unrealistic, uncomprehensive, and incomprehend- 
ible. in the field test version, the school organizes a smaller group of persons who would serve as a 
permanent behavioral objectives task force. They use semi-programmed instructional materials to 
upgrade their expertise, then act as consultants to persons who are responsible for formulating ca- 
reer development units around a goal or set of goals. 
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Career Development Units 

How to plan career development activities posed a special design concern. The career develop- 
ment unit is intended to be a synthesizing activity that brings needs and resource assessment data, 
goals, and objectives into meaningful form for students. It was not the developer's intention to pro- 
vide content for any activity but to provide a framework on which to build career development 
activities. Early versions were sketchy in directing the form of the career development unit to allow 
maximum flexibility for school personnel to create activities. More structure appeared to be needed 
as the first trials resulted in floundering by CDU coordinators. The developers refined the structure 
by stressing the need to orchestrate the development of units with resources to maximize the effect 
on students. Directions for creating a career development unit (CDU) plan became more detailed to 
preclude omitting important consideration to increase the probability of attaining maximum impact. 
A CDU management plan was built to help schools implement multiple efforts in a developmental 
mode. Also built into the framework is an evaluation scheme for each unit which may be used by 
the person leading the unit. 

In providing a framework for CDU development, perhaps the greatest obstacle to overcome was 
and still is the perception that the CDU is not an end in itself. The mechanics of building a good CDU 
could be overwhelming if not managed properly. Career development units should be designed 
in a systematic, comprehensive way in order to effectively upgrade a school's career development 
program. Nevertheless, a CDU plan serves to increase the probability of attaining goals rather than 
guarantee success. 

Installation Concerns 

There are some overall concerns about the design of the product and how well it functions in 
the school environment. The size and scope of the project has a potential for making major changes 
in the school with regard to career development activities. The specified committees necessitated the 
involvement of many faculty, staff, and student members of the school over a period of time. While 
the developers recognized the principal as the key policy maker, it was apparent that a majority of the 
staff and students was needed to support the project. As prerequisite to participation the developers 
recommended a 70 percent or neutral vote from staff and students. 

Another realistic problem that had to be dealt with involved immediate results. The instrumental 
population became easily frustrated by the amount of planning information that had to be gathered. 
It was important to develop student activities as soon as possible to maintain the support of the school. 
Because the product was not fully developed this did not occur at the product engineering sites and 
tended to erode support in the school. In the field test version, the product is designed to enable a 
school to produce student career development activities during the first year with the caveat that 
systematic planning and evaluation is the overall goal of Operation Guidance. 

Three other concerns peculiar to building the product in the school setting are worthy of note. 
The developers had to infuse into the procedures an adaptability factor to attempt to overcome po- 
tential resistance to the product being placed on the school from an outside agency. Thus, the product 
holds to the idea that local needs are best ameliorated by local solutions. Audiovisual elements sup- 
plementing the procedures of the product were difficult to develop. School personnel often reacted 
negatively to crude prototypes of sound slide presentations. Their concern for technical quality or 
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"slickness" tended to distort the message. Production costs, however, prevented heavy investment 
in the technical quality during the early development stages. Lastly, throughout the product engi- 
peering phase, holiday and summer vacations had a near deadly effect on the sites' progress with the 
product. Key activities of the product, the developers found, had to be-avoided immediately before 
and after long breaks. 

The definition of terms and responsibilities created special problems. The product treats the 
improved career development of students as a goal for all professional educators, not just guidance 
counselors, although counselors are viewed as key leaders because of their expertise and existing in- 
volvement in the career development of youth. In some instances counselors felt threatened that 
others would do their work. On the other hand, teachers could not see doing what they perceived 
as the counselors' work. These cannot be resolved by the product itself except by communicating 
as clearly as possible its theoretical approach. The field test version, hopefully, addresses this con- 
cern better than did the earlier versions of the product. 

The developers were also concerned about the availability of technical advice. The intent of 
the product is to be self installable and function without the developers. In each of the six product 
engineering schools a full-time person, trained by the developers, was employed to monitor and lend 
technical guidance to the school. In effect, this person became an advocate who was often a key to 
the success of the project. Generally, the credibility of the project depended on the perceived cred- 
ibility of the field associate. In instances where the person was new to the school, credibility took 
longer to establish. 

The developers realized the full-time field associate would not be a reasonable requirement 
once the product was developed. Through the experience of these six people, the intent was to 
streamline the product so a person already in the school could manage the project on a half-time 
basis. The project coordinator's position description became more concrete over time and can be 
examined in Appendix G. 

Throughout the product engineering phase the developers sought to build a useful product for 
helping high schools improve the career development skills of their students. With the help and co- 
operation of many people at the local, district, and state educational levels, it was possible to build 
from the procedural model, a practical comprehensive career development planning and evaluation' 
package. Although much had been accomplished during the three and one-half years of the work 
unit, the "ultimate" product wc. not attained due to the common lament of developers, insufficient 
resources. The most difficult decision of the developers was to release the product for utilization in 
the next developmental phase, the product field test. 
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V. A NATIONAL FIELD TEST 



Design and Schedule 

Final revisions to the operational planning and evaluation part of the CPSS product were made 
from November 1973 through September 1974 in preparation for a national field test of the proto- 
typic product. The field test is scheduled to last from September 1974 to June 1976. 

Preparation for field testing began in February 1974 with the development of site selection cri- 
teria. Given budgetary limitations, it was determined that from ten to twelve states with three to 
five schools each representatively distributed across the country should constitute the sample. The 
initial sample was to allow for attrition among the participating states and schools. 

Plans called for each chief state school officer, directors of vocational education, and directors 
of guidance being notified of the pending field test even with the relatively late request for partici- 
pation in March 1974. Twenty-nine states positively responded to the initial invitation. From that 
group twelve states were selected by May 1974, based on geographical and population characteristics. 
These twelve states are; Alaska, Arizona, Kentucky, Michigan, Mississippi, Missouri, Montana, New 
Hampshire, North Carolina, Ohio, Utah, and Washington. A lack of resources prevented a wide-scale 
extensive field test. Initially, these states identified forty-nine schools to participate in the two-year 
field test. 

Field Test Purpose 

It is anticipated these schools will generate not only useful data for CVE, but meaningful career 
guidance activities for their students. Although evaluative data for making final revisions to the pro- 
gram planning and evaluation strand will be collected during the field test, the actual revisions, if 
needed, will not be effected until the final synthesizing activity of the Career Planning Support Sys- 
tem. Once the strands presently being developed are integrated, the whole Career Planning Support 
System product should be available in late 1976. 

For readers who would be interested in a general overview of the product and its development, 
A General Report of the Product Engineering of A System for Upgrading High School Career Planning 
Programs is available from Product Utilization Section, The Center for Vocational Education, The 
Ohio State University, 1960 Kenny Road, Columbus, Ohio 43210. 
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Ms. Margaret Jones, East Bank Senior High School, East Bank, West Virginia 
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Ms. Veda M. Syers, Agua Fria Union High School, Avondale, Arizona 
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Paul Bennewitz, Assistant Associate Superintendent for Career Education, Arizona 

Department of Education 
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Vocational Education, Arizona Department of Education 
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The faculty, staff, and students of Agua Fria Union High School 
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Robert L. Collins,Principal, Booker T, Washington High School 
Alvin A. Dawson, former Principal, Booker T. Washington High School 
Roy A. Hendricks, Chairman, State Board of Education 
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John W. Letson, Superintendent, Atlanta Board of Education 
J. Y. Moreland, Superintendent, Area ) 
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Jack P. Nix, Executive Director, State Board of Education 
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Leslie L. Adkins, Director of Student Services, Oregon State Department of Education 
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Department of Occupational Education and Technology, Texas Educational 

Agency 

Bill Lovelace, formerly Educational Program Director, Texas Educational Agency 
Arthur Pace, Principal, Jesse H. Jones High School 

Letitia W. Plummer, Director of Guidance, Houston Independent School District 
J. B. Whiteley, Supervisor, Occupational and Adult Education, Houston Independent 
School District 
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David Acord, Assistant Superintendent for Vocational and Career Education, Kanawha 
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APPENDIX B 

Operation Guidance Product Elements 



1. Steering Committee Handbook 

Written in a procedural and systematic way, this document (536 pages) contains infor- 
mation for t ) management of the planning, implementation, and evaluation of a career de- 
velopment program in a comprehensive high school, grades 9-12. It also includes reference 
copies of product development elements 3, 4, and 7 through 13, below. Each school receives 
two copies. 

2. Staff Development Handbook 

Based upon the use of simulation in learning, this document (500 pages) serves as an 
instructional tool to assist school personnel to understand the concepts of Operation Guidance 
and follow the procedural steps in Operation Guidance. It also contains reference copies of 
elements 4 and 5, below. One copy of this document is provided to each school. 

3. Camera-ready Materials 

These materials (138 pages) are provided as part of the product so that the consumer 
can duplicate instruments, forms, tables, and special instructions that will be used in the local 
educational agency. Two copies are provided to each school. 

4. Career Development Data 

This data (85 pages) consists of information collected on the school and community, the 
results of the student, faculty, parent, and graduate surveys, the identification of existing re- 
sources, such as equipment, materials, space, people, and finances, and a description of career 
education programs presently in operation in the school and community. Two camera-ready 
copies and one loose-leaf binder are provided to each school. 

5. Principal's Handbook 

This document (50 pages) contains an overview of the Operation Guidance process and 
describes the Principal's role in the adoption of Operation Guidance, the orientation of Oper 
ation Guidance to the faculty/staff, and provides specific tools to assist the Principal in review- 
ing goals, behavioral objectives, career development plans, and evaluative program decisions. 
Two copies are provided to each school. 
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6. 



Advisory Committee Handbook 



This handbook (30 pages) is written to provide personnel in the community serving on the 
Operation Guidance Advisory Committee an overview of Operation Guidance and contains tools 
to assist the committee in helping the school plan, implement, and evaluate a career develop- 
ment program. Ten copies are provided to each school. 

7. Resource Assessment Procedural Guide 

This guide (50 pages) contains procedural steps describing the role of the Resource Assess- 
ment Leader and the School and Community Description Task Force in the identification and 
accounting of resources. One copy is provided to each school. 

8. Data Collection Procedural Guide 

Specific procedural instructions are provided in this document (43 pages) so that the Data 
Collection and Tabulation Task Force, consisting of students and faculty, can collect career 
development data on the needs of students, faculty, parents, and graduates. One copy is pro- 
vided to each school. 

9. Data Tabulation Procedural Guide 

This document (64 pages) consists of specific procedural instructions that assist the Data 
Collection and Tabulation Task Force in translating by hand raw data into meaningful data on 
provided tables. Ten copies are provided to each school. 

10. Behavioral Objective Procedural Guide 

This document (30 pages) contains procedural steps to assist behavioral objective special- 
ists (faculty/staff members) to acquire an ability in the writing of behavioral objectives and 
provides useful techniques in assisting Career Development Unit coordinators and advisors to 
derive criterion and enabling objectives from goal statements. Six copies are provided to each 
school. 

11. The Behavioral Objective Manual 

Based upon the concept of programmed learning and self instruction, this manual (53 pages) 
defines the parts of a behavioral objective, types of behaviors that could be included in a be- 
havioral objective, the differences between criterion and enabling objectives, and the process of 
deriving criterion and enabling objectives from goal statements. Six copies are provided to each 
school. 

12. Method Analysis Procedural Guide 

This guide (11 pages) consists of specific procedures to assist personnel in reviewing cur- 
rent and available career development methods applicable in their school. Two copies are pro- 
vided to each school. 
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13. Career Development Unit Procedural Guide 

This procedural guide (22 pages) consists of specific instructions developed to assist school 
personnel in preparing, implementing, and evaluating a Career Development Unit, defined as an 
instructional and/or counseling unit that attempts to achieve a career development goal. Ten 
copies are provided to each school. 

14. AV 1, Operation Guidance: An Overview 

This is a synchronized slide sound presentation (script included) that describes the Oper- 
ation Guidance process and the kinds of people that will be involved in planning, implementation, 
and evaluation. One copy is provided to each school. 

15. AV 2, Your Needs and Resources 

This is a synchronized slide sound presentation (script included) that provides an overview 
of resource assessment, including a description of the School and Community Task Force and 
resource accounting, an overview of the needs assessment including a description of the Data 
Collection and Tabulation Task Force, and the relationship of resource assessment and needs 
assessment to the total scope of work. One copy is provided to each school. 

16. AV 3, Program Goals 

This is a synchronized slide sound presentation (script included) describing the process of 
writing goals based upon the career development needs of the youth in the school, who writes 
goals, and the relationship of sound, well written goal statements to the total career development 
program. One copy is provided to each school. 

17. AV 4, Behavioral Objectives 

This is a synchronized slide sound presentation (script included) describing the process of 
writing behavioral objectives that adequately reflect the goal statements, who writes behavioral 
objectives, and the relationship of good, well written and technically sound behavioral objectives 
to the total career development program. One copy is provided to each school. 

18. AV 5, Career Development Units 

This is a synchronized slide sound presentation (script included) describing the process of 
preparing, implementing, and evaluating Career Development Units, who is involved with Career 
Development Units, and the relationship of Career Development Units to a unified and integrated 
career development program. One copy is provided to each school. 

19. Operation Guidance 

Rubber stamp. 

20. Career Guidance, A Handbook of Methods 

Campbell, Walz, Miller and Krieger. Two copies are provided to each school. 
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APPENDIX C 



Product and Installation Specifications 



Product Specifications 

1.0.0 The instrumental population will be able to state goals for a career planning program that. 

1.1.0 Individually describe student outcomes which tend to ameliorate current stu- 
dent deficiencies in career planning competencies; 

1.2.0 Are collectively responsive to directives of state and local educational agencies 
when available; 

1.3.0 Are individually and collectively understandable by high school students and 
faculty; 

1 .4.0 As a sample, represent the domain of student deficiencies in career planning 
competencies. 

2.0.0 The instrumental population will be able to select a set of program goals for implementing 
a career planning program that, within the resources available in the school and commun- 
ity for career planning assistance, are: 

2.1 .0 Optimally responsive to student needs for career planning competencies, and 
2.2.0 Reflect directives of state and local educational agencies. 
3.0.0 High school students will be able to attain program goals selected for implementation. 

3.1.0 The instrumental population will be able to state behavioral objectives that: 

3.1.1 Are logical derivations of program goal; 

3.1.2 Comprehensively sample the domain of behaviors needed by students 
to attain the program goals (criterion and enabling objectives); 

3.1.3 Describe the competencies needed by the instrumental population 
to facilitate student attainment of desired career development out- 
comes (the enabling objectives); 

3.1.4 Communicate the intended outcomes to students and faculty. 
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3.2.0 The instrumental population will be able to acquire and organize available resources 
(people, space, equipment, materials individually and in combination in the school 
and community) to enable students to achieve goal(s). In order to do so, the instru- 
mental population should be able to : 

3.2.1 Select method(s) that maximize the chances for the specified actors to achieve 
the desired behavior(s), including currency, "face validity," content validity, 
external validity, and feasibility. 

3.2.2 To the degree feasible and desirable, select resources (people, space, equip- 
ment, and materials in the school and community) outside the traditional 
responsibility of the school's pupil personnel services unit. 

3.2.3 Allocate or reallocate available resources (people, space, equipment, and ma- 
terials in the school and community) to maximize the number of goals that 
students have an opportunity to attain. 

3.2.4 Delimit the allocation of resources to only those behavioral objectives that 
can be adequately supported. 

3.2.5 Organize the available resources in such a manner as to maximize the chances 
for students to achieve the selected program goals. 

3.3.0 The instrumental population will be able to install career planning activities into the 
school and/or community as indicated by the following: 

3.3.1 Systematic means by which faculty and students are made aware of the op- 
portunity to attain career planning competencies; 

3.3.2 Satisfaction by participating faculty and students with the career planning 
activity; 

3.3.3 Presence of necessary resources (people, space, equipment, and materials) in 
the school and/or community to accomplish the activity as specified in 3.2.3, 
3.2.4, and 3.2.5, above, when needed. 

3.4.0 The instrumental population will be able to describe the degree to which students achieve 
behavioral objectives and, if not, recommend alternate means which may enable students 
to do so. 



Installation Specifications 

1.0.0 The instrumental population will state that the prototypic Operation Guidance product 
is desirable for their school and that it will have or has had the desired impact on the 
career development of students. 
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1 .1 .0 The instrumetnal population will indicate that the career development of 
students is the responsibility not only of the counselors but of the entire 
facutly and community. 

1.2.0 The instrumental population will state that the effective change of their 
school's career development program can be best effected through long- 
range planning and systematic implementation. 

1.3.0 The instrumental population will state that they will accomplish the prescribed 
evaluations and make their own decisions about applying the results. 

1.4,0 The instrumental population will state that the product will enable them to 
upgrade their career development program through systematic evaluation and 
palnning but is not a career guidance program, per se. 

2.0.0 The instrumental population will organize themselves to accomplish the necessary plan- 
ning and evaluation activities. This includes the formation and directing of a standing 
management group, ad hoc and standing working groups, and an external advisory group. 

3.0.0 The instrumental population will state the need for {deficiency in) career planning and 
implementation competencies experienced by students. 

3.1.0 Given questionnaires, the instrumental population will collect information on 
career planning competencies within the constraints prescribed by the protec- 
tion of human subjects guidelines of the U.S. Department of Health, Education 
and Welfare. 

3.2.0 Given completed questionnaires, the instrumental population will be able to 
compile the data manually or through the use of electronic data processing 
equipment. Manual tabulation will be 95 percent accurate. 

4.0.0 The instrumental population will be able to state program goals. 

4.1 .0 The instrumental population will be able to distinguish between a program 
goal and a non-goal. The program goal will have a student specified as an 
actor and the desired career planning outcome that may or may not be in 
measurable and observable terms. 

4.2.0 The instrumental population will be able to describe a construct for career 
planning competencies from which a program could be organized. 

4.3.0 The instrumental population will be able to expand the construct in response 
to regulations issued by state ancf local educational agencies. 

5.0.0 The instrumental population will be able to rank career development areas in order of 

desired implementation based on the degree to which students need to attain career plan- 
ning competencies, the perception of student need by recent graduates, parents, and 
faculty/staff, and the regulations and resources of state and local educational agencies. 
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6.0.0 The instrumental population will be able to rank program goals in order of desired imple- 
mentation based on the degree to which students need to attain career planning compe 
tencies and based on the regulations and resources of state and local educational agencies. 

7.0.0 The instrumental population w*.!! be able to write behavioral objectives that will have an 
actor, nn observable task the actor is to perform (i.e., behavior that is an outcome rather 
than a process), and criterion(s) to determine mastery of the task. The instrumental popu- 
lation will also demonstrate an understanding of behavioral objectives as follows. 

7.1.0 Given a list of non behavioral objectives, the instrumental population will be 
able to describe the deficiency in each objective without error. 

7.2.0 Given a list of behaviors, the instrumental population will be able to select be- 
haviors that are primarily examples of cognitive, affective, or psychomotor 
behaviors. 



7.3.0 The instrumental population will be able to define criterion and enabling ob- 
jectives. 

8.0.0 The instrumental population will be able to (1) identify and (2) account for resources used 
for implementing career development units. 

9.0.0 The instrumental population will be able to identify career guidance methods or sets of 
methods that will enable the actors (most likely students and faculty) to achieve a be- 
havioral objective or a set of behavioral objectives and that will be supportable by existing 
resources. 



10.0.0 The instrumental population will be able to develop evaluation instruments, including 
administering them and tabulating results from them, in order to assess the effectiveness 
and efficiency of the career development program, 

1 1.0.0 The instrumental population will be able, for a given goal or set of goals, to incorporate 
objectives, methods, resources, and an evaluation strategy into an overall career develop- 
ment plan. The plan should include the desired installation strategy. 

12.0.0 The instrumental population will be able to implement career development units, that is, 
carry out the career development plans, within the school. 

13.0.0 The instrumental population will be able to revise their career development program to 
keep it current and effective, based on evaluative information, 

14.0.0 Personnel 



14.1.0 Individual members of the instrumental population may actively participate as 
often as they desire. However, the gross numbers of personnel required in each 
school are as follows: 
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14.1.1 



Administrators— all. 



14.1.2 Counselors-all. 

14.1.3 Teachers-approximately 21. 

14.1.4 Students-approximately 22. 

14.1 .5 Representatives of the community-approximately 7. 

14.2.0 A Steering Committee member (teacher, counselor, student, or administrator) 
should not be required to spend more than an average of 1.5 hours per week 
working with the product. 

14.3.0 A task force member (teacher, counselor, or student) should not be required 
to spend 2 hours per week for longer than 8 weeks working with the product. 

14.4.0 Each school should need no more than a half-time coordinator to serve as a 
"change agent." 

14.5.0 A customer service representative should not be required to serve more than 20 
schools. 

14.6.0 Total number of people can be scaled downward as the number of students in 
the school decreases from 600. 

15.0.0 Time. A school should be able to complete one developmental cycle, including the pilot 
test of at least two career development units during the first academic year of using the 
product. 

16.0.0 Cost 

16.1.0 During one developmental cycle, a school will consume the following resources and 
services: 

16.1 .1 Printing (offset or comparable method) - the number of pages will be 
approximately 8.5 times the number of students in the school. 

16.1.2 Reproduction - 200 copies. 

16.1.3 Postage (if surveys are mailed) - approximately $100. 

16.1.4 Office supplies (excluding paper for reporduction) - two reams of 
letterhead and 1400 business-size envelopes imprinted with the 
school's name. 

16.1.5 Equipment - a 35 mm circular slide projector with remote control and, 
optionally, a slide-sound synchronizer capability. 

59 



ERIC 



17.0.0 The high school should be stable. Some indicators of the degree to which a school is 
stable are as follows: 

17.1.0 The number and intensity of student, faculty, and community confrontations with 
the school and school district due to racial, socioeconomic, and/or political issues. 

17.2.0 The rate at which school tax levys and bond issues are passed or failed. 

17.3.0 The leadership style of the administration, including the degree to which the 
faculty and students participate in decision making. 

17.4.0 The rate at which innovations are introduced into the school. 

17.5.0 The rate at which the racial composition of the student body and/or faculty 
changes. 

17,6.0 Frequency in changes of administrators, such as the principal, the superintendent, 
and other key administrators at the district level. 

17.7.0 The frequency of administrative changes to the attendance area (geographic area 
in which the student population lives) for the school. 

17.8.0 Number of reorganizations of the school either in the existing facility or due to 
a move to a new facility. 

17.9.0 The rate at which the faculty changes due to retirement, transfer, and/or resig- 
nation. 

18.0.0 The high school shoudi be willing to accept change. This does not imply that the school 
must be an "early adopter" of educational innovations. Some indicators of the degree to 
which a school would be willing to accept innovations are as follows: 

18.1.0 The degree to which faculty and students perceive innovations as succeeding or 
failing, especially those developed by external developers. 

18.2.0 The rate at which faculty and student suggestions and proposals for change are 
accepted and implemented by the administration. 

18.3.0 The rate at which the faculty changes due to transfer and resignation. 

18.4.0 " The rate at which students drop out of school. 

19,0.0 Each school using the prototypic Operation Guidance product will have a change agent 
and a cadre of advocates for the product, including, and probably most important, the 
principal. There should also be at least one advocate at the district and state levels of 
governance with sufficient influence and/or power to persuade key decision makers to 
encourage the adoption of the product. The presence of at least a district level policy 
on implementing accountability is highly desirable. 
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20.0.0 The product (combined product and installation packages) is developed for use by both 
staff and student members of the instrumental population in a variety of schools. There- 
fore, a balance between simplicity and sophistication is needed. 



21.0.0 The instrumental population should be able to synchronize either manually or electronically 
the verbal and visual components of audiovisual materials. 

22.0.0 Participation in Operation Guidance by the instrumental population should not unduly 
disrupt normal school operations. 
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APPENDIX D 



AIDE Plan for Behavioral Objectives 



THE CENTER FOR RESEARCH AND LEADERSHIP DEVELOPMENT 
IN VOCATIONAL AND TECHNICAL EDUCATION 

at 

The Ohio State University 



Program: 



014 



Vocational Development and Adjustment I 
Career Planning Support System 



Component: 



014C 



PLAN FOR DECISION EVENT REVIEW 



Title: 



Operation Guidance: Systems Approach 
for Upgrading Career Guidance Programs 
(Product Engineering) 



Decision Event: 



014Cb1 



Recycling Decision for Module 4 of the 
Operation Guidance Product 



Dates for Review: 



March 6-7, 1974 



Date of Plan: 



January 1974 
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Program: 024, Vocational Development and Adjustment - R. E. Campbell 
Workunit C, Operation Guidance (Product Engineering) • W. N. Suzuki 
Decision Event: 1, Module Four (Behavioral Objectives) 
Major Decision Point: Assessment of the Adequacy of Module Four 
Proposed Date of Review: February 28, March 1, 1974 



Decision Event Plan 



1. Rationale 

Module 4 is a critical link in the Operation Guidance system because it supplies the procedures 
and instructions for the instrumental population (teachers, students, counselors and administrators 
in comprehensive high schools) to develop behavioral objectives that are communicative, critical and 
comprehensive. During Module 4, the behavioral objectives are derived from the school's career guid- 
ance program goals which were identified and validated in the previous module. The objectives de- 
fine and operationalize the career guidance goals, the general directions in which the school should 
move in order to satisfy p .ssing student needs. The objectives also serve as the basis for assessing the 
achievement of program goals and add to the data base and framework for initiating programs to amel- 
iorate salient career development needs. The behavioral objectives are the means for assessing the ade- 
quacy of the Procedural Guides, Manual, and materials given to the schools in Module 4, in directing 
the development of behavioral objectives which meet certain necessary criteria. 

The purpose of the decision event review is to: (1 ) assess or judge the quality of behavioral ob- 
jectives from three of the summative product engineering field trial sites and if appropriate, (2) recom- 
mend necessary revisions to the materials provided to the schools. Findings of the review will be used 
to decide whether the Operation Guidance component is meeting the product specifications and pro- 
vide information for final revision of the module. 



2. Specifications and Criteria 

Behavioral objectives will be assessed in terms of the following major product questions; 

a. Do behavioral objectives contain critical behavioral outcomes (not processes)? 

b. Are behavioral objectives logically related to needs and goals? 

c. Are behavioral objectives clearly stated (communicate)? 

d. Are behavioral objectives technically (ABCD "rule") adequate? 

e. Are behavioral objectives sufficiently representative of the repetoire of behaviors necessary, 
as indicated by their associated goal? 
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In order to determine the adequacy of Module 4 instructions and procedures, a major process 
question must be addressed. Are there alternative ways for a school to produce behavioral objectives 
of equal or better quality at less expense (in terms of time)? 



3. Data Elements 

During the assessment, reviewers will be provided with copies of the following primary and 
secondary data: 

Primary: 

1. Form 2 10, Program Goals. Documentation of the Schools' program goals, which have 
been derived from statements of student needs, and of the behavioral objectives, which 
have been derived from the schools' program goals. 

2. Procedural Guides 4-1 and 4-2 and Manual 4-1. 

3. Report from the review of decision event 014Cal. The report contains the findings of 
and recommendations made by two external reviewers during the assessment of state- 
ments of students needs from three comprehensive high schools serving as field trial sites. 

Secondary: 

1 Manual 2 2, School Characteristics. The documentation of the findings of a needs assess 
ment by each of the field sites. The primary source of data was from a survey of students, 
faculty/staff, parents and graduates. 

2. Form 2 8, Community Characteristics. Documentation of existing information on the 
community, such as labor force information, social service agencies, etc. 

3. Other materials may be useful to the reviewers, especially if they make recommendations 
for revising product elements of Module 4, will be provided upon request. These supple 
mentary materials are: 

a. Project and Product Memorandums. Anecdotal comments, suggestions, problems, etc., 
identified by the on-site project staff members are recorded on these forms. 

b. Project Log. The project logs contain a daily summary of activities at the field trial 
sites. They are prepared by the on-site project staff members. 

4. Conditions 

Considerations pertaining to development of the outputs, e.g., reading difficulty of materials, 
task force membership and sophistication, are as follows: 
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1. Each school is unique. Therefore, goals and behavioral objectives are expected to differ 
among the three schools serving as field trial sites for summative product engineering, 

2. Members of task forces are, as a vvhole, not prepared to derive behavioral objectives prior to 
the use of the Operation Guidance materials, Step-by-step procedures and self-instructional 
materials were provided to assist them with their task. 

3 Procedural guides, instrumentation, and self-instructional materials have been developed 
for use by both faculty and students at comprehensive high schools. Therefore, materials 
may be too simplistic and detailed for some mrmbers of the target population; too sophis- 
ticated for others. Since the materials are in the product engineering stage, major deviations 
from the procedures are discouraged, but schools have been observed deviating from the 
prescribed procedures. 

4, There are ten members on the Behavioral Objectives Task Force, including the chairperson, 
at each of the three field trial sites. Working in teams of three (3), task force members de-' 
velop and refine behavioral objectives for the career guidance goals (Form 2-10) which are 
equally divided among the team. After the objectives are written, the teams convene as a 
whole to review and rate the quality of the entire group of behavioral objectives and refine 
the objectives when appropriate, 

5- The procedures and instrumentation for accomplishing all tasks through the development 
of behavioral objectives have been revised once based on evaluative data from one prior 
field trial of the materials. 



Review Procedures 



At least one week prior to the review, three reviewers selected on the basis of their methodolog- 
ical and substantive expertise in guidance, evaluation and materials development, will 



receive: 



a- Cover letter verifying review request; charges and responsibilities of reviewer(s); date(s) and 
time of review; schedule for review and information on travel, accommodations, and per diem. 

b. Program and workunit abstracts. 

c. Operation Guidance outputs/outcomes. 

d. Organization of school committees. 

e. Summary of data sources for evaluation of the core procedural system. 

f. Decision Event plan including Appendix A: reviewer instructions and Appendix B: 
flow chart of review procedures. 

g. Procedural Guides 4-1 and 4-2. 

h. Manual 4-1. 
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Each of the reviewers will assess the behavioral objectives (Forms 2 10) from two of three 
schools and report his/her judgments and comments on Reviewer Worksheets. If appropriate, re- 
viewers will make recommendations for revisions in Operation Guidance materials. Each reviewer 
will independently prepare a report of his/her findings and recommendations in accordance with 
Attachment A: Format for Reviewer Report. 

It is anticipated that two days are needed for the review, at the end of which time a debriefing 
session will be held. Program personnel will be given an opportunity to ask for clarificaiton of re- 
viewer comments and judgments. (It Is understood that agreement between reviewer (s) and program 
staff is not required). 

6. General Directions for Reviewers 

Step A. Review Procedural Guides 4 1 and 4 2 and Manual 4 1 prior to assessing objectives. 
Step B: Procedures: Review of Behavioral Objectives: 

1. Given a list of criterion and enabling objectives and their associated career development 
goals (Forms 2 10), the reviewers should examine a criterion objective and all its associ- 
ated enabling objectives before proceeding to the next criterion objective. Judgments 
about criticality, logicality , clarity and technical adequacy for each objective as well as 
comments and suggested revisions will be recorded on Reviewer Worksheet Form A for 
an appropriate goal. 

2. After reviewing and judging all the enabling objectives for a particular criterion objective, 
the reviewers should determine if the enabling objectives are comprehensive, that is suf- 
ficient in and of themselves to define the criterion objective or adequately define the crit 
ical behaviors nece^isary to achieve the criterion objective. Judgments about comprehen 
siveness will be recorded on the Reviewer Worksheet. Form A. 

3. The reviewer should examine the next criterion objective and its associated enabling ob 
jectives, concomitantly recording judgments and comments on the Reviewer Worksheet, 
Form A. 

4. The reviewer should determine if the criterion objectives are comprehensive, that is, if 
they are sufficient in and of themselves to define the goal. Judgments and comments con- 
cerning revisions, additions and deletions will be recorded on Reviewer Worksheets. 

5. After reviewing all criterion and enabling objectives for a goal, the reviewer should proceed 
to the next goal and begin a new Reviewer Worksheet, rating objectives as specified in the 
previous steps. 

Step C: Procedures: Review of Materials 
Area of Concern 

1 . The reviewers should examine the ratings of behavioral objectives on Reviewer Worksheets 
Form for patterns or consistencies in objectives not meeting the criteria, criticality, logi- 
cality, clarity, technical adequacy and comprehensiveness. 
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a. If necessary, the reviewers should (1) trace problems back to the appropriate sections 
in Manual 4 1 and Procedural Guides 4-1 and 4-2 and/or (2) determine if problems re- 
flect constraints within the school or community. 

b. If materials are deemed as inadequate for a school and changes are necessary, reviewers 
should make specific reccmmenJjtions for revisions in the materials. Reviewers should 
use Reviewer Worksheet Form b to record comments and recommendations. 



2. The reviewers should examine the criterion and enabling objectives for a school to determine 
if there are problems in distinguishing between criterion and enabling objectives. 

a. If appropriate, the reviewers should trace problems back to the sections in Procedural 
Guides 4-1 and 4-2 and Manual 4-1 . 

b. If changes are necessary in the materials, reviewers should make specific recommenda- 
tions for revisions on Reviewer Worksheet Form b. 

c. Reviewers should determine if a distinction should be made between criterion and en- 
abling objectives using previous ratings and comments as a basis for judgment. 

(1) If the distinction between enabling and criterion objectives should not be made, 
reviewers will provide a rationale for this judgment. 

(2) If a distinction between enabling and criterion objectives should be made, re- 
viewers will provide a rationale for this judgment especially in light of the ade- 
quacy of materials and the ratings on Reviewer Worksheet Form A. If materials 
need to be changed, reviewers should make suggestions for revisions on Reviewer 
Worksheet Form b. 



3. The review procedures in "1 " and "2" above are repeated for the next schools assigned to 
reviewers. The adequacy of materials is assessed on the basis of criterion and enabling ob- 
jectives from each school. 



Reviewers should determine if changes in materials are unique to a school or necessary 
across schools. Reviewers should point out revisions and changes warranted by ratings of 
ofjectives and assessment of materials for both schools. Comments and recommendations 
should be recorded on Reviewer Worksheet Form b. 



Step D: Summary Findings 

1 . The reviewers should provide a summary of findings about the acceptability of behavioral 
objectives and the adequacy of materials. On the Summarv Findings Sheet, reviewers will 
record their judgments and provide a rationale incorporating and summarizing previous 
judgments and comments. 
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Comments: 



1 . The decision event is also an N I E milestone. 

2. The dates of the desired review have not yet been firmly set since the rate at which each 
participating school progresses is self determined. The review is tentatively scheduled for 
February 28 and March 1975. 



Decision Alternatives: 

Based on the review of data elements, each reviewer will be asked to rate whether the objectives, 
as a whole and for each school are acceptable and whether the materials (Procedural Guides and Man- 
ual) as a whole and individually for each school are adequate in instructing and guiding schools to pro« 
duce acceptable objectives. R^yiewers will record their judgments and provide a rationale for each of 
their judgments on the Summary Findings sheet. 

These ratings will be the basis for a decision concerning revisions that may or may not be needed 
in the materials and, if warranted, termination of the workunit. 
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REVIEWERS INSTRUCTIONS 



Introduction 

This guide is intended to be self administering. The procedural steps that follow and the ac- 
companying flow chart (Appendix A) describe the process to be used in determining the quality of 
the behavioral objectives found on or attached to Forms 2-10 (Program Coals). An example Form 
2-10 is attached (Appendix B). ^ 

Before starting the actual assessment of the behavioral objectives, it is suggested that the fol- 
lowing three tasks be accomplished: (a) double check the package of materials to insure that you 
have all the necessary materials: copies of Form 2-10, Reviewer Worksheets, Summary Findings 
sheet. Flow Chart of Review Procedures, Procedural Guides 4-1 and 4-2 and Manual 4-1, (b) care- 
fully read Procedural Guides 4-1 and 4-2 and (c) carefully read Manual 4-1. 

The flowchart in Appendix A provides an overall graphic description of the evaluation process. 
The procedural steps in Section I explicate the tasks outlined in the flowchart. To facilitate the use 
of the flowchart, each procedural step has been numbered according to the corresponding step in the 
flowchart. It is recommended that the procedural steps be used as you go through at least the first 
cycle of evaluation, i.e., the first goal and its related behavioral objectives. As you become familiar 
with the process you may want to use the flowchart and the procedural steps as a reference. 

After you have completed assessment of the behavioral objectives according to the procedural 
steps in Section 1, look at the entire group of behavioral objectives for each school. Determine if 
there is a pattern or consistency in objectives that do not meet Specified criteria. If you find this to 
be the case, determine whether the source of problems is perhaps due to constraints within the school 
(see project logs and information about the school and community) or inadequacies in the Manual and 
Procedural Guides. 

Procedural steps in Section II will guide you in tracing problems with behavioral objectives back 
to the appropriate parts of the Procedural Guides and Manual. Suggest ways to improve the materials 
and provide alternatives to Module 4 procedures. 

After you have completed (1) assessing the behavioral objectives and (2) examining materials as 
warranted by the objectives, you are asked to arrive at some overall conclusions as a culmination and 
synthesis of your previous comments and decisions regarding the behavioral objectives and materials. 
Record your judgments on the Summary Findings sheet and make recommendations for future courses 
of action regarding use of the materials. 
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GLOSSARY 



Behavioral Objective: A statement that describes what a person should be able to do after complet- 
ing a learning experience. Behavioral objectives are written in clear, specific and communicative 
terminology. They describe cognitive, affective or psychomotor behavior in relation to a more 
global goal. 

Clarity: The degree to which a behavioral objective is written in a clear and unambiguous way, the 
extent to which the behavioral objective communicates to its audience. 

Comprehensiveness: The degree to which a cluster of behavioral objectives have included all critical 
behaviors needed to adequately achieve the goal or criterion objective which the particular clus- 
ter is describing. 

Criterion Objective: A behavioral objective which is derived directly from program goals. They de- 
fine thr knowledge, skills, and attitudes that are needed by an actor to achieve goals. Criterion 
objectives are often called "terminal objectives" since they describe only the final or ultimate 
set of knowledge, skills, and attitudes needed by an individual to demonstrate the achievement 
of goals. 

Criticality: The degree to which a behavioral objective is necessary, important, or relevant to describe 
the goal (or criterion objective) which it is describing. 

Enabling Objective: A behavioral objective which specified knowledge, skills, and attitudes that are 
needed by students to achieve a criterion objectives. Enabling objectives are derived from cri- 
terion objectives. 

Goal- A statement written as a possible solution to career development problems or deficiencies ob- 
tained directly from a needs assessment. Goals serve as the basis for describing a career develop- 
ment program, and should specify what students need in order to make the transition from school 
to the world ot work or post»high school education. 

Logicality: The degree to which a behavioral objective is logically derived from the goal (or criterion 
objective) which it describes. 

Technical Adequacy: The degree to which a behavioral objective conforms to the ABCD rule (page 
6 of Manual 4 1); specifically, the presence of a clearly defined actor, a specific behavior ^ an 
adequate statement of conditions (if necessary), and an appropriate degree of success measure. 
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SECTION I: 



Review of Behavioral Objectives 



PROCEDURAL STEPS 
Goals (green) 

1. Read one career guidance goal located on Form 2-10. Record the number of this goal in 
the appropriate space in the Reviewer Worksheet. 



Criterion Objectives (blue) 

2. Read a criterion objective that describes the goal above. Record the number of this cri- 
terion objective in the appropriate space in the Reviewer Worksheet. 

3. Determine whether the criterion objective is critical or not. Ask yourself the following 
question: 

Does this criterion objective describe a necessary and important behavior relevant 
to career development? 

If your answer to the above question was "yes," check the "yes" column under the 
"critical" heading for that particular criterion objective and proceed to step 5 below. 
If, in your estimation, the criterion objective seems unimportant, check the "no" col- 
umn under the "critical" heading for that particular criterion objective. Indicate your 
reasons in the comment section and proceed to step 4. 

4. Answer the following question: 

Is this the last criterion objective for this particular goal? If your answer to the 
above question is "yes," proceed to step 24. If your answer is "no," return to step 2 
above. 

5. Determine if the criterion objective is logical. Answer the following questions: 

Is the criterion objective logically derived from the goal which it defines? Does the 
criterion objective seem to fit or describe the domain of the goal? If, in your judgment, 
the criterion objective is logically derived from the goal, check the "yes" column under the 
"logical" heading for that criterion objective and proceed to step 7. If you believe the cri- 
terion objective does not logically describe the goal, check "no" in the "logical" column 
and proceed to step 6. 
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6. Determine an appropriate goal. Since this criterion objective has been found to be a 
critical behavior but does not logically describe the goal, an alternative goal should be 
identified by one of the two following processes: 

a. Review the list of Career Guidance Goals and determine if the criterion objective 
adequately and logically describes one of these goals. If so, record the number of 
the new goal in the comment section following the particular criterion objective 
in question. 

b. If none of the goals found on the Career Guidance Goal list seem appropriate for 
the criterion objective, create a goal from which the criterion objective logically 
flows. Record this new goal in the comment section. 

7. Determine if the criterion objective is clear. The following questions will help you in this 
decision: 

Is the criterion objective written in a manner that is neither vague nor ambiguous? 

Is the criterion objective written in an understandable way? If the answer is "yes," 
check the "yes" category of the "clear" column and proceed to step 9 below. If the 
answer is "no," check the "no" category of the "clear" column for that particular criter- 
ion objective and proceed to step 8. 

8. Clarify the criterion objective by rewriting any vague or ambiguous portions. Record the 
revised criterion objective in the "comment" section. 

9 Determine if the criterion ofjective is technically adequate according to the ABCD rule. 
(The ABCD rule is discussed in Manual 401, pages 7 19.) The following questions should 
be answered: 

a. Does the criterion objective specify a clearly defined actor? 

b. Does the criterion objective specify an observable behavior? 

c. Does the criterion objective specify an appropriate condition? (A condition is 
optional.) 

d. Does the criterion objective specify a degree of success or a measurable criterion? 

If any of the above questions can be answered "yes," check the "yes" column under 
"technical adequacy" and proceed to step 1 1 . If the answer to all of the four questions 
above is "no," check the "no" column under "technical adequacy" and proceed to 
step 10. 

10. For each question in step 9 that was answered "yes" add the missing part of the criterion 
objective; that is, the actor, behavior, condition, or degree of success, whichever is ap- 
plicable, and record the missing portion(s) in the comment section. 
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Enabling Objectives (yellow) 



11. Determine if any enabling objectives exist for the criterion objective under study. If 
there are any enabling objectives, proceed to step 12. If no enabling objectives exist for 
this particular criterion proceed to step 21. 

12. Read an enabling objective carefully and record the number of the enabling objective in 
the appropriate space on the Reviewer Worksheet. 

13. Deiormine whether the enabling objective is critical or not. Answer the following question: 

Does this enabling objective describe a necessary and important prerequisite behavior 
needed to achieve the criterion objective which it defines? 

If your answer is "yes," check the "yes" category of the "critical" column for that par- 
ticular enabling objective and proceed to step 14. If your answer to the above question 
is "no," check the "no" response under the "critical" heading for that particular enabling 
objective and proceed to step 20. 

14. Determine if the enabling objective is logical. Answer the following questions: 

Is the enabling objective logically derived from the criterion objective which it 
describes? Does the enabling objective seem to fit or define the domain of the criterion 
objective? 

If, in your judgment, the enabling objective is logically derived from the criterion ob- 
jective, check the "yes" response under the "logical" heading for that particular enabling 
objective and proceed to step 16. If you believe the enabling objective is not a necessary 
prerequisite to achieve the criterion objective, check the "no" response. 

15. Suggest an appropriate criterion objective. Since this enabling objective has been found 
to be a critical prerequisite behavior, but is not logically derived from the criterion ob- 
jective, an alternative criterion objective must be formulated by one fo the following 
processes: 

a. Review other criterion objectives describing the goal in question and decide if the 
enabling objective is a necessary prerequisite for another criterion objective. If so, 
record the number of the criterion objective in the "comment" section. 

b. Suggest a criterion objective which describes the goal being analyzed for which the 
enabling objective becomes a logical prerequisite. Write this new criterion objective 
in the "comment" section, 

16. Determine if the enabling objective is clear. The following questions will help you in this 
decision: 
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Is the enabling objective written in a manner that is neither vague 
nor ambiguous? Is the enabling objective written in an understandable 
way? 

If the answer to the above questions is "yes" check the "yes" response in the "clear" 
column and proceed to step 18. If the answer is "no" check the "no" response and 
proceed to step 1 7. 

17. Suggest ways in which the enabling objective could be clarified by rewriting any vague 
or ambiguous portions. Record the revised enabling objective in the "comment" section. 

18. Determine if the enabling objective is technically adequate according to the ABCD 
rule. The following questions should be answered: 

a. Does the enabling objective specify a clearly defined actor? 

b. Does the enabling objective specify an observable behavior? 

c. Does the enabling objective specify an appropriate condition? 

d. Does the enabling objective specify a degree of success 
or measurable criterion? 

If any of the above questions can be answered "yes," check the "yes" response 
under the "technical adequacy" column and proceed to step 20. If the answer 
to all of the four questions above is "no," check the "no" response and proceed 
to step 19. 

19. Suggest ways in which the enabling objective could be technically improved. Record 
the revised enabling objective in the "comment" section. 

20. Answer the following question: 

Is this the last enabling objective for this particular criterion objective? 

If your answer to the above question is "yes" proceed to step 21 . 
If your answer is "no," return to step 12 above. 

21. Determine the comprehensiveness uf the enabling objectives. Reread the goal and 
criterion objective from which the enabling objective(s) are derived. In your opinion, 
are there any critical prerequisite behaviors or enabling behaviors that may have been 
overlooked? If you believe that all critical enabling objectives have been identified, 
place a check in the "yes" response in the "comprehensive" column related to the 
set of enabling objectives being studied and proceed to step 23. If any other enabling 
objectives are needed for the criterion objective, check the "no" column under 
"comprehensive" and proceed to step 22. 
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REVIEW OF BEHAVIORAL OBJECTIVES 
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22. Suggest any additional prerequisite behavior, or preferably, enabling objectives on a 
separate sheet of paper for the particular set of enabling objectives in question. Make 
sure that the number of the goal and the criterion objective from which the new enabling 
objective is derived are also indicated on the paper. 

23. Answer the following question: 

Is this the last criterion objective for this particular goal? If your answer to the above 
question is "yes," proceed to step 24. If your answer is "no," return to step 2 above. 

24. Determine the comprehensiveness of the criterion objectives. Reread the goal from which 
the criterion objective!?) are derived. In your opinion, are their any critical criterion ob- 
jectives that may have been overlooked? If you believe that all critical criterion objectives 
have been identified, place a check in the "comprehensive" column related to the set of 
criterion objectives oeing studied and proceed to step 26. If any other criterion objectives 
are needed to define th goal adequately, check the "no" column under "comprehensive" 
and proceed to slap 25. 

25. Record any additional behaviors, or preferably, criterion objectives on a separate sheet of 
paper for the particular set of criterion objectives in question. Indicate also any enabling 
objectives which may be needed to describe any prerequisite behaviors necessary for the 
achievement of any new criterion objectives that have been identified. Be sure to indicate 
the number of the goal from which the new criterion objective is derived. 

26. Answer the following question: 

Is this the last goal in your set of Program Goals (Form 2-10)? 

If the answer to this question is "no," return to step 1 above. If the answer is "yes," you 
have completed your task. You may want to write down any overall comments regarding 
the contents of the behavioral objectives you've just evaluated. 
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Review of Materials 



PROCEDURAL STEPS 

1. Examine the entire group of behavioral objectives for one school. Determine if there are 
patterns or consistencies in objectives not meeting one or more of the criteria, criticality, 
logicality, clarity, technical adequacy and comprehensiveness on the Review Worksheets 
forms (see glossary for operational definitions of the criteria). 



a. If an unreasonable number of behavioral objectives seem to consistently contain 
uncritical behavioral outcomes , examine Procedural Guides 4-1 and 4-2, Manual 401, 
page 28 32. Look for omissions, ambiguities, insufficiencies, etc. in the materials 
that might be contributing factors to objectives not meeting this criterion. Identify 
community and school characteristics (available for each school from program per- 
sonnel) which indicate or verify that the behaviors are not critical and suggest be- 
haviors that might be considered critical. How can the manual and procedural guides 
be revised so that schools will develop behavioral objectives containing critical career 
development behaviors? Use Reviewer Worksheet form b to record your comments 
and specific recommendations. 

b. If an unreasonable number of behavioral objectives are not logically derived ; that is, 
criterion objectives are not logically derived from goals and enabling objectives are 
not logically derived from criterion objectives, refer to Procedural Guides 4*1 and 4«2, 
Chapter V pages 35-37 in Manual 4 1 and Forms 2«10, program goals. Determine if 
problems are due to directions and guidelines in the materials or perhaps attributable 
to constraints within the school. How can the materials be changed so that schools 
can better derive goals and objectives? Use Reviewer Worksheet form b to record your 
comments and specific recommendations. 



c. If an unreasonable njmber of behavioral objectives are not clear, refer to Procedural 
Guides 4 1 and 4 2 and pages 6 21 in Manual 4-1. How can the materials be changed 
so that behavioral objectives are more clearly communicated? Use Reviewer Worksheet 
form b to report your comments and specific recommendations. 



d. If an unreasonable number of behavioral objectives are not technically adequate 
(follow the ABCD rule), determine if deficiencies are due to the materials: Pro- 
cedural Guides 4-1 and 4-2 and pages 6-21 in Manual 4-1. How can the materials 
be changed so that schools can produce behavioral objectives that are technically 
adequate? Use Reviewer Worksheet form b to record your comments and specific 
recommendations. 



78 



082 



ERIC 



e. If an unreasonable number of objectives are not comprehensive to sufficiently define 
a goal or a criterion objective, determine if problems are due to deficiencies in the 
Procedural Guides or constraints in the school. How can the materials be changed 
so that objectives are representative of behaviors necessary to achieve the goal and/or 
objective? Use Reviev^er Worksheet form b to record your comments and specific 
recommendations. 

2. Repeat the above procedures for the second school to which you have been assigned. The 
adequacy of materials is assessed in the basis of criterion and enabling objectives from each 
school. 

3. Examine the criterion and enabling objectives for each school to determine if there are prob- 
lems in distinguishing between criterion and enabling objectives. Record any comments or 
specific recommendations on Reviewer Worksheet Form b. 

4. Determine if changes in materials are unique to a school or necessary across both schools. 
You may confer with the other reviewers to determine degree of change and substantiate 
your specific recommendations. Record your comments and specific recommendations on 
Reviewer Worksheet form b. 

5. Summarize your findings about the acceptability of behavioral objectives overall and for 
each school on the Summary Findings sheet in your packet of materials. Provide a brief 
rationale for your judgments as a summary of your previous comments and judgments. 

6. Summarize your findings about the overall adequacy of the Procedural Guides and Manual 
together in instructing the schools (as a group) in producing acceptable b.o.'s. Then rate the 
adequacy of Procedural Guides 4 1,42 and Manual 4^1 individually in instructing each school 
separately. 
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Section II: 
Materials 



Reviewer 
School (s) 



Reviewer Worksheet Form b 



General Area of Concern: 



Specific Recommenddtions (if possible, reference page numbers and step numbers in the Procedural 
Guides and Manual) 
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Reviewer 



Date 



Summary Findings 

I. Behavioral Objectives 

A. Overall, the behavioral objective for both schools are: 

( ) Acceptable ' ( ) Acceptable with revisions 

B. The objectives developed by 
1. School 



are: 



( ) Acceptable 
2. School 



are: 



( ) Acceptable 
C. Rationale: 



( ) Unacceptable 



( ) Acceptable with revisions ( ) Unacceptable 



( ) Acceptable with revisions ( ) Unacceptable 



II. Materials 

A. Overall, the materials (PG 4-1 and 4-2, Manual 4-1) are: 

( ) Adequate ( ) Adequate with revisions ( ) Inadequate 

B1. ^Procedural Guide 4-1 for School is: 

{ ) Adequate ( ) Adequate with revisions ( ) Inadequate 

B2. Procedural Guide 4-1 for school is: 

{ ) Adequate ( ) Adequate with revisions ( ) Inadequate 

B3. Procedural Guide 4-1 for School is: 

( ) Adequate ( ) Adequate with revisions { ) Inadequate 
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B4. Procedural Guide 4-2 for school is: 

( ) Adequate ( ) Adequate with revisions ( ) Inadequate 

B5. Procedural Guide 4-2 for school Is: 

( ) Adequate ( ) Adequate with revisions | ) Inadequate 

B6. Procedural Guide 4-2 for school is: 

( ) Adequate ( ) Adequate with revisions ( ) Inadequate 

87. Manual 4-1 for school is: 

( ) Adequate ( ) Adequate with revisions ( ) Inadequate 

B8. Manual 4-1 for school is: 

( ) Adequate ( ) Adequate with revisions ( ) Inadequate 

b9. Manual 4-1 for school is: 

( ) Adequate ( ) Adequate with revisions ( ) Inadequate 

C. Rationale: 



III. Recommendation 

A. Overall 

B. Procedural Guide 4-1 

C. Procedural Guide 4-2 

D. Manual 4-1 
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APPENDIX E 

Product Engineering Phase Position Description 
Field Associate 



Activities : 

Establish and maintain liaison between school and school district and The Center, 
Collect evaluative data through test and survey instruments. 
Collect evaluative data through observation and interviews. 
Analyze evaluative data. 

Monitor individual and group performance on Center-developed materials 
and procedures. 

Assist school to use materials and procedures, if needed. 
Revise Center-developed materials and procedures as needed. 
Prepare progress and incident reports; 

Perform administrative functions, such as keep financial records, plan meetings, 
and direct secretary. 

Requirements: 

Education: Plaster's degree in guidance and/or counseling, educational technology, 
ducational research and development, or educational evaluation. 

Educational Experience: 

1. Minimum two years teaching or guidance counseling at high school level; 

2. Participated in the planning, development, and evaluation of educational 
products using a systems approach, including the derivation of behavioral 
objectives. 
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Personal Characteristics: 

1. Ability to: 

a. Observe unobtrusively and assess accurately group and individual 
behavior. 

b. Receive and develop further information, from others, including 
criticism; does not react to criticism strongly or quickly or with 
defensive or submissive behavior. 

c. Interact with students, teachers, counselors, and administrators 
individually or in groups. 

d. Analyze information objectively. 

e. Make decisions rapidly and objectively. 

f. Communicate information to groups. 

g. Follow a task through to its completion independently and 
systematically. 

h. Follow directions precisely. 

i. Moderate individual conflicts. 

j. Recognize problems, develop alternative solutions, and specify 
consequences for each alternative solution. 

k. Handle confidential information. 

2. Familiar with: 

a. Community 

b. Organizational structure and administrative procedures 
of the school and school district. 

3. Willing to travel to The Center for training. 
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APPENDIX F 

National Field Test Position Description 
School Coordinator for Operation Guidance 



71)16 [jusition description is intended only as an aid to selecting the school coordinator and it 
i6 bdbed Oil The Center's .hree years of experience working with individuals in comparable positions. 
It io uui bdief tliat persons who generally meet the requirements below will have the best chance of 
dealing successfully with the demands of the position. 

Role and Responsibilities 

The iehuol coordinator will be closely involved with the staff, students, and community persons 
Hi one hiyh school of the cooperating LEA for at least two years. The coordinator will occupy a vital 
position ds the Steering Committee chairperson and will work in this capacity at least half time, year 
round. The Steering Committee oversees all the Operation Guidance activities and makes final recom 
uiijnJdlio.is to the school principal for final decision based on the data gathered and processed through 
Opctdtion Guidance procedures. Therefore, it is essential that the coordinator clearly understand the 
Optratiofi Guidance product and process. He or she should also be a resource person to the school 
for career guidance rhethods that are reported in the professional literature. 

The coordinator will have a specific leadership position within the school and will assist counsel- 
ui'S, tedthjfb, administrators, students, and persons in the community as they implement the various 
phases of the Operation Guidance package. This person must be skilled in working with individuals 
dfjd yroupb and supporting them in their efforts. If necessary, the coordinator must be willing to 
liikt appropriate action needed to resolve these problems. The school coordinator should be a person 
Aho engages in thoughtful consideration before making decisions but should also be comfortable with 
the responsibility of making rapid decisions when necessary. 

Requirements 
1. Essential 

a. Ability to establish rapport with local education agency personnel and 
state education agency personnel. 

b. Enthusiastic commitment to the need for career development programs 
and the provision of effective career development services to students. 
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c. A minimum one year of experience as a guidance counselor or in other 
positions requiring familiarity with career development, such as career 
education coordinator, etc. 

d. A minimum three years experience as an educational professional in an 
LEA. 



e. A master's degree with study areas relevant to career development theories 
and practices (e.g., career guidance, career education, counseling psychology, 
vocational counseling, etc.). 

f. Experience making presentations to groups, including teachers and students. 

g. In cases where existing state certification standards apply to this position 
and those standards exceed these, state standards shall take precedence. 

2. Desirable 

a. Experience in the school selected for participation in the product field test 
and knowledge of the target community. 

b. Experience directing or coordinating developmental activities, especially 
those involving adults. 

c. Work experience in non-educational pccupations, especially in occupations 
that normally do not require a baccalaureate or higher degree for entry. 

d. Expertise in methods of research, evaluation, systems technology, and 
change processes as they apply to educational institutions. 

e. Experience in curriculum development. 

f. Knowledge of and experience with the derivation and use of behavioral 
objectives. 

3. Other Desirable Personal Characteristics and Skills 

a. Success in this position may depend very much on the person's ability to 
maintain and cultivate good professional relationships with others. 
Consider some of these characteristics: 

(1) Open-minded. 

(2) Confident decision maker. 

(3) Supportive of others. 

(4) Handles conflict diplomatically and ^skillfully. 
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The person should have the ability to identify past, present, and potential 
problems and to hypothesize alternate solutions creatively. 

The person should be able to function as a keen and tactful observer and 
assess behaviors accurately. 

The person should be flexible and tolerant of ambiguity and diversity. 
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